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Introduction 



TIMSS and PIRLS 201 1 : Relationships 
Among Reading, Mathematics, and 
Science Achievement— Implications for 
Early Learning 


Ina V.S. Mullis and Michael O. Martin 
Boston College 

Background Descriptions of TIMSS and PIRLS 

TIMSS (Trends in International Mathematics and Science Study) 
is an international assessment of mathematics and science at the 
fourth and eighth grades that has been conducted every four 
years since 1995, with the most recent assessment in 2011. In 
total, more than 600,000 students participated in TIMSS 2011. 
Countries and regional benchmarking entities could participate 
in the fourth grade assessment, the eighth grade assessment, 
or both: fifty-two countries and seven benchmarking 
entities participated in the fourth grade assessment, and 
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45 countries and 14 benchmarking entities participated in the eighth grade 
assessment. Also, several countries, where fourth and eighth grade students 
were expected to find the TIMSS assessments too difficult, assessed their sixth 
and ninth grade students. 

The TIMSS 2011 achievement results were reported in two companion 
publications, the TIMSS 2011 International Results in Mathematics (Mullis, 
Martin, Foy, & Arora, 2012) and the TIMSS 201 1 International Results in Science 
(Martin, Mullis, Foy, & Stanco, 2012). These reports summarized mathematics 
and science achievement at the fourth and eighth grades, documented trends 
in achievement over time for participants in previous TIMSS assessments, 
and related achievement to the rich array of information about students’ 
characteristics and attitudes as well as their home, school, and classroom 
contexts for learning. 

PIRLS (Progress in International Reading Titeracy Study) is an 
international assessment of reading comprehension at the fourth grade that 
has been conducted every five years since 2001. In total, approximately 325,000 
students participated in PIRLS 2011, including countries assessing students 
at the sixth as well as the fourth grades, regional participants or language 
benchmarking efforts, and prePIRLS (an easier version of PIRLS) for students 
who are still developing their reading skills. Forty-five countries assessed fourth 
grade students in 201 1. 

The PIRLS 2011 results were published in PIRLS 201 1 International Results 
in Reading (Mullis, Martin, Foy, & Drucker, 2012). This report, which is similar 
to the TIMSS 2011 volumes for mathematics and science, contains the 2011 
reading achievement results for the participating countries and benchmarking 
entities, shows trends over time for the countries and benchmarking entities 
that also participated in previous assessments, and relates reading achievement 
to a number of home, school, and classroom contexts for learning to read. 
Full details of the methodology underpinning TIMSS and PIRLS in 2011 are 
presented in Methods and Procedures in TIMSS and PIRLS 2011 (Martin & 
Mullis, 2012). 

Both TIMSS and PIRLS 2011 continue series of international assessments 
in mathematics, science, and reading conducted by the International Association 
for the Evaluation of Educational (IEA). IEA pioneered international 
comparative assessments of educational achievement in the 1960s to gain a 
deeper understanding of the effects of policies and practices across countries’ 


TIMSS AND PIRLS 201 1 RELATIONSHIPS REPORT 

INTRODUCTION 



TIMSS & PIRLS 

International Study Center 

Lynch School of Education, Boston College 



different systems of education. TIMSS and PIRLS are directed by IEA’s TIMSS 
& PIRLS International Study Center at Boston College. 

Introduction to the Current Report 

In 2011, the TIMSS and PIRLS data collection schedules came into alignment 
for the first time in the history of these international assessments. This provided 
countries with the opportunity to assess their fourth grade students in three 
fundamental curricular areas: mathematics, science, and reading. However, 
more pertinent to the present report, 34 countries and three benchmarking 
entities took advantage of this unique opportunity to assess the same students 
in all three subjects. Equally important, because the PIRLS assessment includes 
a parent questionnaire that provides information describing students’ home 
environments and supports for learning, this home environment information 
was available for the first time with TIMSS data as well. Taken together, 
the fourth grade students in these 34 countries and three benchmarking 
participants have achievement data in the three core academic areas — reading, 
mathematics, and science — accompanied by an extensive array of background 
questionnaire data about the home, school, and classroom contexts for learning 
these three subjects. 1 

Having data on the same students makes it possible to conduct a range of 
investigations of the important characteristics of home and school influencing 
early learning, while controlling for extraneous factors. Researchers can apply a 
variety of modeling techniques to explore these important issues by examining 
the interrelationships among their underlying components. To facilitate this 
research, the TIMSS & PIRLS International Study Center created a special 
international database including only fourth grade students assessed in all three 
subjects, and achievement scores in reading, mathematics, and science were 
estimated based on a multidimensional scaling or reading, mathematics, and 


1 The TIMSS 201 1 fourth grade mathematics and science assessment frameworks were organized around a content 
dimension (number, geometric shapes and measures, and data display in mathematics; life science, physical science, 
and earth science in science), and a cognitive dimension (knowing, applying, and reasoning for both mathematics 
and science). Given the frameworks broad coverage, the assessment item pools were necessarily large — 175 items 
in mathematics, and 217 in science — with about half being multiple choice and half constructed response. TIMSS 
2011 also collected extensive information about students' home supports and school environments for learning. The 
questionnaires given to students, teachers, schools, and parents yielded nearly 20 context questionnaire scales about 
learning and teaching mathematics and science. 

PIRLS assesses two purposes for reading that account for most of the reading done by students in and out of school: 
for literary experience and to acquire and use information. Within each of these two major purposes four comprehension 
processes are assessed: retrieving, inferencing, integrating, and evaluating. PIRLS gives students reading passages 
(texts) approximately 800 words in length and asks them 13-16 questions about each passage. PIRLS 2011 contained 10 
passages (5 for each purpose) and 135 questions in total. The PIRLS achievement scale was used to summarize students' 
performance on the assessment questions. PIRLS also included questionnaires given to students, teachers, schools, and 
parents that were developed in parallel to those administered with TIMSS. Like TIMSS, the PIRLS background data yielded 
nearly 20 context questionnaire scales about students' attitudes toward reading as well their supports and instructional 
experiences in learning to read. 
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science together (Foy, 2013). The purpose of the special database is to have the 
most appropriate basis for studying relationships among reading, mathematics, 
and science teaching and learning. The three separate international reports 
referenced above, and the separate international databases for TIMSS 2011 
mathematics and science (Foy, Arora, & Stanco, 2013) and for PIRLS 2011 
reading (Foy & Drucker, 2013) should be used for information about the results 
in one or another of the three subjects assessed in 201 1. 

It is anticipated that the primary value of this special TIMSS and PIRLS 
2011 data will be realized through in-depth national research, as participating 
countries use the data for school improvement at the primary level. The intention 
of this initial book examining relationships among reading, mathematics, and 
science teaching and learning is to illustrate the potential of the special TIMSS 
and PIRLS 2011 database and to make some headway in the analysis process. 
To this end, the book includes four very different analyses as described in the 
following sections. 

Profiles of Achievement in Reading, 

Mathematics, and Science 

In examining the relationships among students’ achievement in reading, 
mathematics, and science, a good starting place is to look at whether primary 
schools are providing students with a thorough grounding in these core subjects, 
and establishing a solid foundation for later learning. The first chapter in the 
book sets the stage for the following three chapters by examining patterns 
of achievement in reading, mathematics, and science within each of the 
34 countries and 3 benchmarking entities. 

For each TIMSS and PIRLS 2011 participant, achievement is profiled at 
the TIMSS and PIRLS High International Benchmark and Low International 
Benchmark, by providing the percentages of students reaching these benchmarks 
in all three subjects as well as in each of the three subjects separately. 2 The data 
also are shown graphically in displays that simultaneously show the results in 
all three subject areas. These graphics show at a glance which countries are 
most successful in educating their fourth grade students to high levels, and 
whether countries are equally successful across all three subjects. Interestingly, 
most countries are more successful in one or two of the subjects than another, 
especially when it comes to educating substantial percentages of students to 
high levels. 


2 The Advanced, High, Intermediate, and Low International Benchmarks are specific points on the TIMSS and PIRLS 
achievement scales. As described in Chapter 1, TIMSS and PIRLS use a scale anchoring procedure to describe what 
students scoring at these benchmarks know and can do. 
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Students performing at the High International Benchmarks in all three 
subjects are very accomplished fourth grade students — able to read relatively 
complex materials with in-depth understanding, solve a variety of mathematics 
problems, and show familiarity with a range of scientific information. These 
students have developed an extremely solid basis for further learning and are 
well positioned to take advantage of future educational opportunities. However, 
the TIMSS and PIRLS 2011 data provide evidence that it is a very challenging 
task to educate students to the level of the high benchmarks by the fourth grade. 
Only Singapore had more than half of its students reach the high benchmark 
in all three subjects, and only two more countries, Chinese Taipei and Finland, 
had at least half of their fourth grade students reach the high benchmark in 
each subject separately. 

More than half the countries, however, were successful in educating 90 
percent of more of their students to the Low International Benchmark in all 
three subjects. These students showed that they can read and comprehend 
facts, read a variety of simple graphs and tables, know simple mathematics 
(such as adding, subtracting, and basic geometric figures), and know science 
facts about health, ecosystems, and animals. Although these students have 
lower achievement than those at the high level, they do have a well-rounded 
foundation in core concepts and skills that provides a good basis for further 
learning. In comparison, students who have not learned the basic fundamentals 
of reading, mathematics, and science by the end of their fourth year of schooling 
may be at some risk for future academic success. 

The profiles across countries of the percentages of fourth grade students 
reaching high and basic levels of achievement help to situate countries with 
respect to their relative performance in reading, mathematics, and science. In 
addition, these profiles provide a good foundation for considering the results 
of the more complicated analyses presented in the subsequent three chapters. 

Impact of Reading Ability on Mathematics and Science 
Achievement: An Analysis by Item Reading Demands 

The TIMSS 2011 Assessment Frameworks (Mullis, Martin, Ruddock, O'Sullivan, 
& Preuschoff, 2009), developed collaboratively with the participating countries 
through a series of reviews, describe the mathematics and science content 
and cognitive processes that were to be assessed. Both the mathematics and 
science frameworks require assessing rather sophisticated reading demands 
even at the fourth grade. For example, topics in the mathematics and science 
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content domains specify that students should be able to solve routine and 
non-routine problems set in everyday contexts and conduct inquiries about 
various phenomena. Understanding the descriptions of the situations for these 
types of problems necessarily involves reading. Moreover, in mathematics, 
the Data Display content area is based on “reading and interpreting” tables, 
pictographs, bar graphs, and pie charts as well as creating such data displays. The 
science framework requires comprehending descriptions of experiments and 
investigations as well as a variety of models and diagrams. Finally, and perhaps 
most importantly, both mathematics and science can be generally regarded as 
specialized languages with their own technical vocabularies. 

In developing items to assess student achievement in mathematics and 
science, TIMSS makes every effort to avoid unnecessary reading so that the 
language used is no more complex than necessary to frame the question (and 
responses for multiple choice items). However, inevitably the assessment items 
in both mathematics and science assessments vary considerably in the reading 
demands they place on students. Reading requirements can be quite minimal, 
as in items requiring students to complete a calculation or identify the smallest 
or largest quantity, and most of these items are short. However, some items 
can have more substantial reading demands, as in those requiring students to 
understand a description of a science experiment or phenomenon and then 
apply their knowledge or explain their reasoning. 

The availability of PIRLS 2011 data on reading achievement provided an 
ideal opportunity to investigate the relationship between reading ability and 
the reading demands of the TIMSS 2011 fourth grade mathematics and science 
assessment items. Fourth grade students are likely to be at a disadvantage in 
learning mathematics and science as well as demonstrating high performance 
on the TIMSS assessment if they lack reading skills. 

Essentially, the study examined the hypothesis that students with high 
reading ability would not be impacted by the level of reading demands in the 
TIMSS items, but that poorer readers would score lower on the items with 
highest reading demands than on the items with the lowest reading demands. 

Essentially, a coding scheme was used to categorize the TIMSS mathematics 
and science items into groups (low, medium, and high) in terms of the reading 
demands they place on the student. The coding scheme evaluated each item in 
terms of length, technical vocabulary, and density of graphical displays (pictorial 
representations, models, tables, and graphs). The basic approach used in the 
analysis was to examine, for each participating country and benchmarking 
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participant, the relationship between fourth grade students’ reading ability as 
measured by PIRLS and their performance on TIMSS items with increasing 
levels of reading demands. 

The methods used to evaluate the reading demands of the items provided 
additional insight into the TIMSS items from the perspective of “mathematics 
reading” and “science reading.” Although the total number of words was the 
strongest factor, technical vocabulary and complicated diagrams also contribute 
to reading demands. The most significant contribution, however, was gaining a 
deeper understanding of the interconnectedness among curriculum coverage, 
instructional emphasis, cognitive processing, and reading ability and the 
challenges in trying to disentangle the various roles they have in affecting 
student achievement. The results varied from country to country and even 
between mathematics and science within countries, yet there was support for the 
idea that more reading demands can make the fourth grade TIMSS items more 
challenging for weaker readers even in the context of variation in curriculum 
coverage and that assessing more complex cognitive processing often involves 
more reading. 

What are the Characteristics of Effective Schools in 
Reading, Mathematics, and Science? 

In order to address this question, school effectiveness analyses were conducted 
to study what makes schools successful, beyond having a majority of students 
in attendance from advantaged socioeconomic backgrounds. From an analytic 
perspective, school effectiveness studies make use of multilevel modeling in 
order to analyze the relationship between school factors and achievement after 
controlling for the influences of students’ home backgrounds. 

The research in this chapter began with a strong conceptual model of 
school effectiveness based on the existing body of school effectiveness research 
and the factors that influenced school quality as documented in the TIMSS 
2011 and PIRLS 2011 International Reports. According to the conceptual 
model, an effective school was safe and orderly, supported academic success, 
had adequate facilities and equipment, was staffed with well-prepared teachers, 
had well-resourced classrooms, and provided effective instruction. From the 
vast amount of contextual background data available in TIMSS and PIRLS 2011, 
eventually eleven context questionnaire scales were combined into five robust 
school effectiveness measures that were available in parallel across reading, 
mathematics, and science: three measures of effective school environment, and 
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two measures of effective school instruction. The Home Background Control 
model also included two measures: the Home Resources for Learning scale, 
and an index of students’ ability to do numeracy and literacy tasks when they 
started school. 

Separately for each country — and for reading, mathematics, and science 
within each country — a series of multilevel regression models were formulated. 
These models were used to describe how the school explanatory measures 
were associated with achievement, both before and after controlling for home 
background at the student and school level. 

Although there was variation from country to country, the Home 
Background Control model was successful in capturing the relationship 
between home background and students’ achievement in reading, mathematics, 
and science in every country, with the Home Resources for Learning variable 
the strongest predictor. In fact, 16 of the participants had just one significant 
predictor after controlling for home background. 

The school variables posited by the conceptual model were positively 
correlated with student achievement in most countries, providing support 
for the validity of the model. After controlling for home background, of the 
school environment variables, Schools Are Safe and Orderly was related 
to higher achievement in at least one subject in 15 countries, and Schools 
Support Academic Success in 10 countries. Students Engaged in Reading, 
Mathematics, and Science Lessons was the most powerful school instruction 
variable, related to higher achievement in at least one subject in 15 countries, 
again after controlling for home background. All in all, a school that was safe 
and orderly, promoted academic excellence, and provided engaging instruction, 
could be considered to have several important characteristics for effectiveness. 

It should be realized, however, that countries with little or no differences 
from school to school in student achievement (including at least seven in 
this research) provide little scope for an effective school analysis of the type 
described here. Factors such as the ones considered in this research are still 
important school factors for supporting high student achievement, but an 
analysis focused on differences between schools cannot show evidence of their 
effects. 

Home Support for Literacy and Numeracy Achievement 

One of the most stable findings in educational research is the impact of 
students’ background on achievement, especially parents’ level of education 
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and occupation or earnings. Also, a great deal of research on child development 
has highlighted the importance of home environments that stimulate the 
development of early literacy skills. Consistent with this research, in each 
PIRLS assessment cycle the PIRLS home background data have shown a strong 
positive association between student reading achievement at the fourth grade 
and home educational resources, parents’ emphasis on early literacy activities, 
and childrens literacy skills when entering school. 

Although there has been less research conducted about early numeracy 
skills, this is an area of growing interest. Therefore, the PIRLS 2011 Home 
Background Questionnaire, which was administered to parents of students who 
participated in both TIMSS and PIRLS, was designed also to collect data on early 
numeracy activities and children’s numeracy skills upon entering school. The 
literacy and numeracy background data, in association with students’ reading, 
mathematics, and science achievement, provide an excellent opportunity to 
examine the differential effects of aspects of home environment on student 
achievement in these essential subjects. 

The fourth research study presented in this book, conducted by Jan- 
Eric Gustafsson, Kajsa Yang Hensen, Monica Rosen from the University of 
Gothenburg in Sweden, took particular advantage of the information about 
children’s early literacy and numeracy experiences provided by the Home 
Background Questionnaire. This research adopted a path modeling approach 
to investigate the extent of the influence of Parental Education and Gender 
on mathematics, science, and reading achievement at the fourth grade; and 
the mechanisms through which Parental Education and Gender influence 
achievement via books in the home, frequency of early literacy and numeracy 
activities, and the child’s ability to carry out literacy and numeracy tasks when 
starting school. The variables in the model were ordered chronologically and 
logically. In general, Parental Education and Gender preceded the number of 
books in the home, which preceded the literacy and numeracy activities with 
the pre-school child, which preceded the child’s early literacy and numeracy 
skills at the beginning of first grade, which preceded the PIRLS 2011 reading 
achievement and TIMSS mathematics and science achievement scores at the 
fourth grade. 

In the first step of estimation, a common model was fit based on the 
pooled data from all 34 countries and three benchmarking entities, after 
which separate models were fit for each country. In the pooled data, the 
total effects of Parental Education were substantial for mathematics, science, 
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and reading (.33, .35, and .35, respectively), and books in the home was an 
important mediating variable. The common model provided strong support 
for the hypothesized chain of influence via books, early activities, and ability 
entering school to achievement. The number of books was related to frequency 
of activities in the home oriented towards both literacy and numeracy, and 
these activities influenced the general levels of literacy and numeracy skills the 
child had developed at the time of entering school. Interestingly, a stronger 
emphasis on early literacy activities than on numeracy activities influenced both 
the levels of children’s literacy and numeracy skills when entering school as 
well as their achievement in the fourth grade. Similar results have been found 
in other studies, perhaps because adequate language skills are a prerequisite for 
learning mathematics. In comparison, the effects of Gender were much weaker. 
There were essentially no gender differences in mathematics or science, although 
the total effect on reading achievement was rather substantial (.12). Also, only 
a small part of the Gender effect was mediated via the variables in the model, 
although for girls the early learning activities were oriented more toward literacy 
than numeracy. 

While the overall findings and mechanisms described above were identified 
in most countries, there were interesting differences across the countries. There 
is much additional TIMSS 2011 and PIRLS 2011 background data that can be 
used to expand this research, and the research can be extended in many different 
directions to investigate further variables and hypothesized mechanisms. 

Summary 

In summary, the TIMSS and PIRLS 2011 fourth grade combined database 
provides an important resource for researching the contexts for early in reading, 
mathematics, and science. The achievement measures are extremely robust and 
there is a rich array of context questionnaire data. It is hoped that the four 
chapters of this books will inspire many others to conduct further research and 
mine this valuable data. 
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Chapter 1 


Profiles of Achievement Across 
Reading, Mathematics, and Science 
at the Fourth Grade 


Ina V.S. Mullis 
Boston College 


Overview 

TIMSS routinely reports about students’ achievement in 
mathematics and science, and PIRLS routinely reports about 
achievement in reading. However, the cycles of the two 
assessments coinciding in 201 1 made it possible for countries to 
have the same fourth grade students participate in both TIMSS 
and PIRLS. Thirty-four countries and three benchmarking 
participants took advantage of this opportunity to collect 
internationally comparable reading, mathematics, and science 
achievement on the same fourth grade students together with a 
large amount of background data. 
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Most relevant for this chapter, having TIMSS and PIRLS achievement data 
on the same students enables a comparison of achievement across the three 
subject areas in each country, although such a comparison has its challenges. 
This research addresses the question: 

Are primary schools around the world providing students 
a solid foundation in core subjects — reading, mathematics, 
and science? 

The chapter presents profiles of fourth grade achievement across reading, 
mathematics, and science for each of the 34 countries and three benchmarking 
participants. Because both excellence and equity are important educational goals 
for countries around the world, achievement is profiled at the high level and 
also at the basic level. For a healthy citizenry and economy, it is important to 
have fourth grade students well prepared in reading, mathematics, and science 
concepts so that they can take full advantage of their further educational 
opportunities, and it also is important to understand how many students have 
a grasp of the basics across reading, mathematics, and science, as well as how 
many are lagging behind and still struggling with elementary skills and concepts. 
Those lagging behind may be at risk for academic success in the future. 

The TIMSS and PIRLS achievement results at the fourth grade, as reported 
separately, suggest that some countries are remarkable in the high levels of 
achievement their students attain in particular subjects. For example, the East 
Asian countries, including Singapore, Korea, Hong Kong SAR, Chinese Taipei, 
and Japan excel in mathematics from assessment cycle to assessment cycle, and 
the Russian Federation and Finland are top performers in reading (please see 
TIMSS 201 1 International Results in Mathematics and PIRLS 201 1 International 
Results in Reading). This raises the question: Are fourth grade students receiving 
a well-rounded education across the core subject areas, or is there less emphasis 
on some areas in some countries? 

It is well known that performance on the TIMSS and PIRLS achievement 
scales cannot be compared directly in terms of the content they represent 
(i.e., a “tablespoon” of mathematics achievement, for example, does not equal 
a “tablespoon” of reading achievement). However, the TIMSS and PIRLS 
International Benchmarks do provide a basis for comparisons from subject to 
subject, because they define the same points on each subject’s achievement scale 
in terms of what students performing at those points know and do in reading, 
mathematics, or science. 
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TheTIMSS and PIRLS 201 1 International 
Benchmarks at the Fourth Grade 


The TIMSS and PIRLS achievement scales summarize students’ performance 
on large numbers of test items designed to measure breadth of understanding 
and cognitive processing in mathematics, science, and reading, respectively. 
At each grade, the achievement results are reported on the mathematics, 
science, and reading achievement scales, each with a range of 0 — 1,000 
(although student performance typically ranges between 300 and 700). In 
each of the three subjects in addition to average achievement, TIMSS and 
PIRLS report achievement at four points along the scales as international 
benchmarks: Advanced International Benchmark (625), High International 
Benchmark (550), Intermediate International Benchmark (475), and Low 
International Benchmark (400). The percentage of students reaching each of 
these international benchmarks provides information to a country on student 
achievement all across the achievement spectrum. The TIMSS & PIRLS 
International Study Center worked with its subject matter expert advisory 
committees, the PIRLS 2011 Reading Development Group and the TIMSS 
2011 Science and Mathematics Item Review Committee, to conduct detailed 
scale anchoring analyses to describe achievement at the benchmarks in reading, 
mathematics, and science, respectively. In a scale anchoring analysis, the 
students’ achievement on the items in the assessment is used to identify what 
knowledge and skills are associated with achievement at particular points on 
the achievement scale. For example, fourth grade students scoring at the High 
International Benchmark (550) in mathematics were likely to solve an algebra 
problem requiring reasoning, whereas students scoring at lower levels on the 
scale were much less likely to answer this problem correctly. 

In every participating country, TIMSS and PIRLS can identify the students 
that reached each of the various international benchmarks, and so it was decided 
to use the data on students reaching the high and low international benchmarks 
to conduct the analyses presented herein. The High International Benchmark 
was selected for this study rather than the Advanced International Benchmark, 
because only small percentages of students (if any in some countries) reached 
the advanced level. 

Exhibit 1 . 1 contains the descriptions of students’ achievement in reading, 
mathematics, and science at the High International Benchmarks. The High 
International Benchmark represents a proficient or competent level of 
fourth grade achievement in each subject and provides an interesting point 
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of comparison from country to country. Although the 50 countries that 
participated in TIMSS 201 1 at the fourth grade did not intersect completely with 
all 45 countries that participated in PIRLS 2011, the median percentages (half 
the countries above and half below) of students reaching the high benchmarks in 
TIMSS and PIRLS 2011, were 28 percent in mathematics, 32 percent in science, 
and 44 percent in reading (indicating countries had somewhat less difficulty 
reaching the High International Benchmark in reading than in mathematics 
or science). 

Exhibit 1.1: Descriptions of High International Benchmarks of 
Achievement at the Fourth Grade 


Reading 


When reading literary texts, students can locate and distinguish significant actions 
and details embedded across text; make inferences to explain relationships between 
intentions, actions, events, and feeling, and give text-based support; interpret and 
integrate story events and character actions and traits from parts of texts; evaluate the 
significance of events and actions across an entire story; and recognize the use of some 
language features (e.g., metaphor, tone, imagery). When reading informational texts, 
students can locate and distinguish relevant information with a dense text or a complex 
table; make inferences about logical connections to provide explanations and reasons; 
integrate textual and visual information to interpret the relationship between ideas; and 
evaluate content and textual elements to make a generalization. 


Mathematics 


Students can apply their knowledge and understanding to solve problems. They can 
solve word problems involving operations with whole numbers, and use division in 
a variety of problem situations. They can use their understanding of place value to 
solve problems, and extend patterns to find a later specified term. They demonstrate 
understanding of line symmetry and geometric properties. Students can interpret and 
use data in tables and graphs to solve problems, and use information in pictographs and 
tally charts to complete bar graphs. 


Science 


Students can apply their knowledge and understanding of the sciences to explain 
phenomena in everyday and abstract contexts. They demonstrate some understanding 
of plant and animal structure, life processes, life cycles, and reproduction. They also 
demonstrate some understanding of ecosystems and organisms' interactions with 
their environment, including understanding of human responses to outside conditions 
and activities. Students demonstrate understanding of some properties of matter, 
electricity and energy, and magnetic and gravitational forces and motion. They show 
some knowledge of the solar system, and of Earth's physical characteristics, processes, 
and resources. Students demonstrate elementary knowledge and skills related to 
scientific inquiry. They compare, contrast, and make simple inferences, and provide brief 
descriptive responses combining knowledge of science concepts with information from 
both everyday and abstract contexts. 


Exhibit 1.2 contains the descriptions of students’ achievement in reading, 
mathematics, and science at the Low International Benchmarks. The Low 
International Benchmark indicates basic proficiency or competence. It is 
very important for students’ future school careers to have developed a solid 
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foundation of basic understandings and skills across the core subject areas by 
the early grades. Students not reaching the Low International Benchmarks 
in one or more core subjects may be at some risk for future success in their 
educational careers, and may fall farther and farther behind their peers as they 
continue in school. Again, somewhat different countries participated in TIMSS 
2011 at the fourth grade than in PIRLS 2011, but the median percentages of 
fourth grade students reaching the low benchmarks in TIMSS 2011 and PIRLS 
2011, were 90 percent in mathematics, 92 percent in science, and 95 percent 
in reading. The data across all participating countries indicate a high degree 
of success in educating students in basic concepts and skills, across reading, 
mathematics, and science. 

Exhibit 1.2: Descriptions of Low International Benchmarks of 
Achievement at the Fourth Grade 


Reading 


When reading literary texts, student can locate and retrieve an explicitly stated detail. 
When reading informational texts, students can locate and reproduce explicitly stated 
information that is at the beginning of the text. 


Mathematics 


Students have some basic mathematical knowledge. Students can add and subtract 
whole numbers. They have some recognition of parallel and perpendicular lines, familiar 
geometric shapes, and coordinate maps. They can read and complete simple bar graphs 
and tables. 


Science 


Students show some elementary knowledge of life, physical, and earth sciences. They 
demonstrate knowledge of some simple facts related to human health, ecosystems, and 
the behavioral and physical characteristics of animals. They also demonstrate some basic 
knowledge of energy and the physical properties of matter. Students interpret simple 
diagrams, complete simple tables, and provide short written response to questions 
requiring factual information. 


Looking across the descriptions of achievement at the High International 
Benchmarks in reading, mathematics, and science presented in Exhibit 1.1, it 
can be seen that students performing at the High International Benchmarks in 
all three subjects were very accomplished fourth grade students — able to read 
complex materials with in-depth understanding, solve a variety of problems 
in mathematics, and show familiarity with a range of scientific information. 
In comparison, looking across the descriptions of achievement at the low 
benchmark presented in Exhibit 1.2, it can be seen that students reaching only 
the low benchmark showed that they can read and comprehend facts, read a 
variety of simple graphs and tables, know simple mathematics (such as adding, 
subtracting, and basic geometric figures), and know science facts about health, 
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ecosystems, and animals. Although these students had lower achievement than 
those at the high level, they do have a well-rounded foundation in core concepts 
and skills that provides a good basis for further learning. 

Profiles of Achievement Across the 
International Benchmarks 

For each country, data are provided about the percentages of fourth grade 
students reaching the PIRLS 2011 High International Benchmark in reading, 
the TIMSS 2011 High International Benchmark in mathematics, and the TIMSS 
2011 High International Benchmark in science, as well as the percentage of 
students reaching the High International Benchmark in all three subjects. 
Students that reached the high benchmark in all three subjects would be 
proficient in reading, mathematics, and science; and very well-equipped to 
pursue more advanced study in a variety of subject areas. 

Similarly, data are provided for each country and benchmarking participant 
showing the percentages of students reaching the Low International Benchmarks 
in each of the subjects, as well as the percentage reaching the low benchmark 
in all three subjects. Countries that have educated most of their fourth grade 
students to the low benchmark in all three subjects are to be congratulated, 
because essentially no students are being “left behind.” A certain degree of equity 
has been achieved, because all students can continue in their schooling, building 
upon their basic foundation of knowledge and skills across the core curriculum 
areas. 

For each country, the percentages of students reaching the benchmarks in 
each subject are presented together with graphic illustrations known as radar 
charts (or star charts). These types of charts are used to plot the values of 
different categories — in this case, the three percentages of students reaching 
the high benchmarks in reading, mathematics, and science — along a separate 
axis in the same graph, with the value of each point represented as the distance 
from the center of the chart. Depicting the data in this way illustrates the 
relative strengths and weaknesses across the three subjects, with the strengths 
depicted by results farther from the center of the graph. As hypothesized based 
on achievement results reported separately from subject to subject, there are 
interesting differences across countries, in that some have considerably higher 
percentages reaching the benchmarks in one or another of the subjects. That 
is, in some countries students reach considerable higher levels of achievement 
in mathematics, for example, than they do in science or reading, while in other 
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countries students are achieving at considerably higher in reading, than in 
mathematics or science. 

Exhibits 1.3 through 1.39 contain the country-by- country results, ordered 
from highest to lowest according to the percentage of students reaching the 
High International Benchmark in all three subjects — reading, mathematics, and 
science. 

Singapore was the only country that had more than half its students 
reaching the High International Benchmark in all three subjects. Two other 
countries, Chinese Taipei and Finland, had 50 percent or more of their students 
reaching each benchmark separately, but they were not the same students. 
Chinese Taipei had 40 percent of its students reaching the high benchmark in all 
three subjects and Finland had 39 percent as did Hong Kong SAR, followed by 
the Russian Federation with 35 percent. All the other participants in this study 
had fewer than 30 percent of their students reaching the high benchmark in all 
three subjects, providing evidence that this is a very challenging educational 
task. The percentages were very small in a number of countries. 

As would be anticipated, more countries had success in raising most 
students to the Tow International Benchmark in all three subjects. More than 
half the countries had 90 percent or more of their fourth grade students reaching 
the high benchmark in all three subjects. 
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Countries at the Fourth Grade 


SINGAPORE The Singaporean fourth grade students showed a particular strength 
in mathematics, with 78 percent reaching the high benchmark, although 
achievement was also very good in science (68%), and in reading (62%). More 
than half the students (54%) reached the High International Benchmark in all 
three subjects, and essentially all of them (95%) reached the Low International 
Benchmark in all three subjects. 

CHINESE TAIPEI The fourth grade students in Chinese Taipei also showed a 
particular strength in mathematics, with about three-fourths (74%) reaching the 
High Benchmark. Again, however, achievement also was very good in the other 
two subjects, with more than half reaching the high benchmark in reading (55%) 
and in science (54%). Forty percent reached the High International Benchmark 
in all three subjects, and essentially all of the students (96%) reached the Low 
International Benchmark in all three subjects. 

HONG KONG SAR Of the countries included in this study, Hong Kong SAR had 
the greatest percentage (82%) of students reaching the High International 
Benchmark in mathematics, and mathematics was a considerable strength. In 
comparison, two-thirds reached the high benchmark in reading, and less than 
half (46%) in science. Still, performance in all three subjects was very good, with 
39 percent of the students reaching the high benchmark in all three subjects, 
and virtually all (97%) reaching the low benchmark. 

FINLAND In comparison to the three previous East Asian countries, the high 
performing Finnish students did less well in mathematics than in science and 
reading. More than three-fifths of the fourth grade students reached the high 
benchmark in science (65%) and reading (63%), and half reached that level 
in mathematics. Thirty-nine percent reached the high benchmark in all three 
subjects, and virtually all (97%) reached the low benchmark in all three subjects. 

RUSSIAN FEDERATION The fourth grade students in the Russian Federation 
demonstrated their particular excellence in reading, and also performed well 
in mathematics and science. The percentages of students reaching the High 
International Benchmark were 63% in reading, compared to 52% in science 
and 47% in mathematics. Thirty-five percent reached the high benchmark in 
all three subject and essentially all students (96%) reached the low benchmark 
in all three subjects. 
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Exhibit 1.3: Singapore 

Profiles of High and Low Performance in Reading, Mathematics, and Science 


TIMSS & PIRLS 
2011 


^th 

Grade 


High International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 

Subjects I 

Percent of Students ! 

All Three Subjects 

54 (1.9) 

Reading 

62 (1.8) 

Mathematics 

78 (1.4) 

Science 

68 (1.7) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 


In At Least One Subject but Not All Three Science 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects 1 

Percent of Students j 

All Three Subjects 

95 (0.6) 

Reading 

97 (0.4) 

Mathematics 

99 (0.2) 

Science 

97 (0.4) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 


In At Least One Subject but Not All Three 


Science 


Reading 



Mathematics 
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SOURCE: lEA's Trends in International Mathematics and Science Study and Progress in International Reading Literacy Study -TIMSS and PIRLS 201 1 










Exhibit 1.4: Chinese Taipei 

Profiles of High and Low Performance in Reading, Mathematics, and Science 


TIMSS & PIRLS 
2011 


^th 

Grade 


High International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 

Subjects 

Percent of Students 1 

All Three Subjects 

40 (1.3) 

Reading 

55 (1.3) 

Mathematics 

74 (1.1) 

Science 

54 (1.3) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 


In At Least One Subject but Not All Three Science 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects 

Percent of Students 1 

All Three Subjects 

96 (0.4) 

Reading 

98 (0.4) 

Mathematics 

99 (0.2) 

Science 

97 (0.4) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 



Science 


Reading 



Mathematics 
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Exhibit 1.5: Hong Kong SAR 

Profiles of High and Low Performance in Reading, Mathematics, and Science 


TIMSS & PIRLS 
2011 


^th 

Grade 


High International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 

Subjects I 

Percent of Students 1 

All Three Subjects 

39 (1.8) 

Reading 

67 (1.6) 

Mathematics 

82 (1.3) 

Science 

46 (2.1) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 


In At Least One Subject but Not All Three Science 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects 1 

Percent of Students j 

All Three Subjects 

97 (0.5) 

Reading 

99 (0.2) 

Mathematics 

100 (0.1) 

Science 

97 (0.4) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 


In At Least One Subject but Not All Three 


Science 


Reading 



Mathematics 
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SOURCE: lEA's Trends in International Mathematics and Science Study and Progress in International Reading Literacy Study -TIMSS and PIRLS 201 1 










Exhibit 1.6: Finland 

Profiles of High and Low Performance in Reading, Mathematics, and Science 


TIMSS & PIRLS 
2011 


^th 

Grade 


High International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 

Subjects 1 

Percent of Students 1 

All Three Subjects 

39 (1.3) 

Reading 

63 (1.2) 

Mathematics 

50 (1.4) 

Science 

65 (1.7) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 


In At Least One Subject but Not All Three SciGITCe 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 2011 
Low International Benchmarks 

Subjects 1 

Percent of Students ] 

All Three Subjects 

97 (0.5) 

Reading 

99 (0.2) 

Mathematics 

98 (0.4) 

Science 

99 (0.3) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 


In At Least One Subject but Not All Three 


Science 


Reading 



Mathematics 
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Exhibit 1.7: Russian Federation 


TIMSS & PIRLS 
2011 


^th 

Grade 


Profiles of High and Low Performance in Reading, Mathematics, and Science 

High International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 

Subjects I 

Percent of Students 1 

All Three Subjects 

35 (1.9) 

Reading 

63 (1.6) 

Mathematics 

47 (2.1) 

Science 

52 (2.0) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 


In At Least One Subject but Not All Three Science 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects I 

Percent of Students [ 

All Three Subjects 

96 (0.6) 

Reading 

99 (0.3) 

Mathematics 

97 (0.5) 

Science 

98 (0.4) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 

I In At Least One Subject but Not All Three r . 

“ Science 


Reading 



Mathematics 
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NORTHERN IRELAND These students performed very well in all three subjects, 
although relatively less so in science. While nearly three-fifths of the students 
reached the high benchmarks in mathematics (59%) and in reading (58%), one- 
third did in science (34%). Twenty-nine percent reached the high benchmark 
in all three subjects, and 92 percent reached the low benchmark in all three 
subjects. 

HUNGARY The Hungarian students performed similarly in all three subjects. Just 
under half the fourth grade students reached the high benchmark in reading 
(48%) and science (46%), while 37 percent did so in mathematics. Twenty-eight 
percent reached the high benchmark in all three subjects. Ninety percent or 
more reached the low benchmark in each of the three subjects, and 88 percent 
reached this level in all three subjects. 

IRELAND In Ireland, the fourth grade students demonstrated a particular strength 
in reading, with 54 percent reaching the high benchmark, compared to 41 
percent in mathematics and 35 percent in science. One-fourth of the students 
reached the high benchmark in all three subjects, and 90 percent reached the 
low benchmark in all three subjects. 

GERMANY The German fourth grade students performed similarly across the 
three subjects, with 46 percent reaching the high benchmark in reading, 39 
percent in science, and 37 percent in mathematics. Nearly one-fourth (23%) of 
the students reached the High International benchmark in all three subjects, 
and most students (94%) reached the Low International Benchmark in all three 
subjects. 

PORTUGAL In Portugal, there were achievement differences across the subjects, 
but no large gaps. Forty-seven percent of the students reached the high 
benchmark in reading, 40 percent in mathematics, and 36 percent in science. 
Similar to Germany, nearly one-fourth (23%) of the students reached the high 
benchmark in all three subjects, and most students (93%) reached the low 
benchmark in all three subjects. 
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Exhibit 1.8: Northern Ireland 

Profiles of High and Low Performance in Reading, Mathematics, and Science 


TIMSS & PIRLS 
2011 


^th 

Grade 


High International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 

Subjects 1 

Percent of Students I 

All Three Subjects 

29 (1.4) 

Reading 

58 (1.4) 

Mathematics 

59 (1.4) 

Science 

34 (1.6) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 


In At Least One Subject but Not All Three Science 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects I 

Percent of Students | 

All Three Subjects 

92 (0.8) 

Reading 

97 (0.5) 

Mathematics 

96 (0.6) 

Science 

94 (0.8) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 


In At Least One Subject but Not All Three 


Science 


Reading 



Mathematics 
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SOURCE: lEA'sTrends in International Mathematics and Science Study and Progress in International Reading Study -TIMSS and PIRLS 201 1 










Exhibit 1.9: Hungary 

Profiles of High and Low Performance in Reading, Mathematics, and Science 


TIMSS & PIRLS 
2011 


^th 

Grade 


High International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 

Subjects 1 

Percent of Students 1 

All Three Subjects 

28 (13) 

Reading 

48 (1.5) 

Mathematics 

37 (1.4) 

Science 

46 (2.0) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 


In At Least One Subject but Not All Three Science 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects 

Percent of Students j 

All Three Subjects 

88 (1.1) 

Reading 

95 (0.8) 

Mathematics 

90 (0.9) 

Science 

93 (0.9) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 

I In At Least One Subject but Not All Three r . 

“ Science 


Reading 



Mathematics 


TIMSS AND PIRLS 201 1 RELATIONSHIPS REPORT 

28 CHAPTER 1 



TIMSS & PIRLS 

International Study Center 

Lynch School of Education, Boston College 


SOURCE: lEA's Trends in International Mathematics and Science Study and Progress in International Reading Literacy Study -TIMSS and PIRLS 201 1 












Exhibit 1.10: Ireland 

Profiles of High and Low Performance in Reading, Mathematics, and Science 


TIMSS & PIRLS 
2011 


^th 

Grade 


High International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 

Subjects 1 

Percent of Students 1 

All Three Subjects 

25 (1.5) 

Reading 

54 (1.4) 

Mathematics 

41 (1.6) 

Science 

35 (1.7) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 


In At Least One Subject but Not All Three Science 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects 1 

Percent of Students 1 

All Three Subjects 

90 (0.9) 

Reading 

97 (0.5) 

Mathematics 

94 (0.6) 

Science 

92 (0.8) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 

| In At Least One Subject but Not All Three 


Science 


Reading 



Mathematics 
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SOURCE: lEA's Trends in International Mathematics and Science Study and Progress in International Reading Literacy Study -TIMSS and PIRLS 201 1 











Exhibit 1.11: Germany 

Profiles of High and Low Performance in Reading, Mathematics, and Science 


TIMSS & PIRLS 
2011 


^th 

Grade 


High International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 

Subjects 

Percent of Students 1 

All Three Subjects 

23 (1.3) 

Reading 

46 (1.3) 

Mathematics 

37 (1.4) 

Science 

39 (1.5) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 



Science 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects 

Percent of Students [ 

All Three Subjects 

94 (0.8) 

Reading 

98 (0.4) 

Mathematics 

97 (0.5) 

Science 

96 (0.8) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 


In At Least One Subject but Not All Three 


Science 


Reading 



Mathematics 
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Exhibit 1.12: Portugal 

Profiles of High and Low Performance in Reading, Mathematics, and Science 


TIMSS & PIRLS 
2011 


^th 

Grade 


High International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 

Subjects 

Percent of Students 1 

All Three Subjects 

23 (1.7) 

Reading 

47 (1.8) 

Mathematics 

40 (2.0) 

Science 

36 (2.0) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 



Science 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects 1 

Percent of Students [ 

All Three Subjects 

93 (1.1) 

Reading 

98 (0.5) 

Mathematics 

97 (0.7) 

Science 

95 (0.9) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 

I In At Least One Subject but Not All Three r . 

“ Science 


Reading 



Mathematics 
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AUSTRALIA In reading, 42 percent of the Australian students reached the high 
benchmark and 35-36% reached this level in mathematics and science. Twenty- 
two percent reached the high benchmark in all three subjects. Although more 
than 90 percent reached the low level in each of three subjects separately, 
somewhat less (86%) reached the low benchmark in all three subjects. 

CZECH REPUBLIC The fourth grade students in the Czech Republic demonstrated a 
relative weakness in mathematics. Although half reached the high benchmark in 
reading and 45 percent in science, a comparative smaller percent (30%) reached 
this level in mathematics. Twenty-one percent of the students reached the high 
benchmark in all three subjects, and most (92%) reached the low level in all 
three subjects. 

SLOVAK REPUBLIC Similar to the Czech students, the Slovak fourth graders 
also demonstrated a relative weakness in mathematics, with 44 percent 
reaching the high benchmark in both reading and science but only 30 percent 
in mathematics. Also, like the Czech students, 21 percent reached the high 
benchmark in all three subjects. Although more than 90 percent of the Slovak 
students reached the low benchmark in each of the subjects, slightly fewer (89%) 
reached it all three subjects. 

LITHUANIA Similar to both the Czech Republic and the Slovak Republic, 21 
percent of the Lithuanian students reached the high benchmark in all three 
subjects. However, the Lithuanian students showed their relative weakness in 
science. Forty-three percent reached the high benchmark in mathematics and 39 
percent in reading, but somewhat fewer (31%) in science. Most students (92%) 
reached the low benchmark in all three subjects. 

SLOVENIA The Slovenian students had the highest percentage of students (42%) 
reaching the high benchmark in reading, the next highest in science (36%), 
and the lowest in mathematics (31%). One-fifth the students reached the high 
benchmark in all three subjects, 90 percent reached the low benchmark in all 
three subjects. 
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Exhibit 1.13: Australia 

Profiles of High and Low Performance in Reading, Mathematics, and Science 


TIMSS & PIRLS 
2011 


^th 

Grade 


High International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 

Subjects 

Percent of Students 1 

All Three Subjects 

22 (1.2) 

Reading 

42 (1.2) 

Mathematics 

35 (1.4) 

Science 

36 (1.4) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 


In At Least One Subject but Not All Three Science 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects 

Percent of Students i 

All Three Subjects 

86 (1.1) 

Reading 

93 (0.7) 

Mathematics 

91 (0.9) 

Science 

92 (0.9) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 


In At Least One Subject but Not All Three 


Science 


Reading 



Mathematics 
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Exhibit 1.14: Czech Republic 


TIMSS & PIRLS 
2011 


^th 

Grade 


Profiles of High and Low Performance in Reading, Mathematics, and Science 

High International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 

Subjects 1 

Percent of Students 1 

All Three Subjects 

21 (1.1) 

Reading 

50 (1.4) 

Mathematics 

30 (1.6) 

Science 

45 (1.5) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 

| In At Least One Subject but Not All Three 


Science 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects 

Percent of Students 

All Three Subjects 

92 (0.8) 

Reading 

98 (0.5) 

Mathematics 

93 (0.8) 

Science 

96 (0.7) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 

I In At Least One Subject but Not All Three r . 

“ Science 


Reading 



Mathematics 
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Exhibit 1.15: Slovak Republic 

Profiles of High and Low Performance in Reading, Mathematics, and Science 


TIMSS & PIRLS 
2011 


^th 

Grade 


High International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 

Subjects 

Percent of Students 1 

All Three Subjects 

21 (1.3) 

Reading 

44 (1.4) 

Mathematics 

30 (1.6) 

Science 

44 (1.6) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 

| In At Least One Subject but Not All Three 


Science 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects 

Percent of Students j 

All Three Subjects 

89 (1.3) 

Reading 

96 (0.8) 

Mathematics 

91 (1.3) 

Science 

94 (1.0) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 


In At Least One Subject but Not All Three 


Science 


Reading 



Mathematics 
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Exhibit 1 . 16 : Lithuania 

Profiles of High and Low Performance in Reading, Mathematics, and Science 


TIMSS & PIRLS 
2011 


^th 

Grade 


High International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 

Subjects 

Percent of Students 1 

All Three Subjects 

21 (1.3) 

Reading 

39 (1.3) 

Mathematics 

43 (1.6) 

Science 

31 (1.6) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 



Science 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects 

Percent of Students [ 

All Three Subjects 

92 (0.8) 

Reading 

97 (0.4) 

Mathematics 

96 (0.6) 

Science 

95 (0.6) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 


In At Least One Subject but Not All Three 


Science 


Reading 



Mathematics 
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Exhibit 1.17: Slovenia 

Profiles of High and Low Performance in Reading, Mathematics, and Science 


TIMSS & PIRLS 
2011 


^th 

Grade 


High International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 

Subjects 

Percent of Students j 

All Three Subjects 

20 (1.3) 

Reading 

42 (1.3) 

Mathematics 

31 (1.3) 

Science 

36 (1.6) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 


In At Least One Subject but Not All Three Science 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects 

Percent of Students j 

All Three Subjects 

90 (0.7) 

Reading 

96 (0.5) 

Mathematics 

94 (0.6) 

Science 

94 (0.6) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 


In At Least One Subject but Not All Three 


Science 


Reading 



Mathematics 
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ITALY The Italian fourth grade students show considerable variation in 
achievement across the three subjects. In reading, 46 percent of the students 
reached the high benchmark, in science 37 percent, and in mathematics 28 
percent. Eighteen percent reached the high benchmark in all three subjects, and 
90 percent reached the low benchmark in all three subjects. 

SWEDEN The Swedish students showed a relative weakness in mathematics. 
In reading, 47 percent of the students reached the high benchmark and in 
science 44 percent did, but in comparison only 25 percent reached that level 
in mathematics. Similar to Italy, 18 percent reached the high benchmark in all 
three subjects, and 91 percent reached the low benchmark in all three subjects. 

AUSTRIA Very similar to Sweden, the Austrian fourth grade students also showed 
a relative weakness in mathematics. In science, 42 percent of the students 
reached the high benchmark and in reading 39 percent did. However, only 26 
percent reached that level in mathematics. Eighteen percent reached the high 
benchmark in all three subjects, and 92 percent reached the low benchmark in 
all three subjects. 

ROMANIA Seventeen percent of the Romanian fourth grade students reached the 
high benchmark in all three subjects, with 37% reaching that level in science, 
32% in reading, and 28% in mathematics. Considering that the percentage of 
students reaching the high level in all three subjects was similar to a number of 
the preceding countries, it is interesting that only 73 percent reached the low 
benchmark in all three subjects. 

CROATIA The Croatian fourth grade students showed considerable variation 
in achievement across the three subjects. They had very good achievement in 
reading, with more than half the students (54%) reaching the high benchmark. 
Thirty percent reached the high benchmark in science, but only 19 did in 
mathematics. While 13 percent reached the high benchmark in all three 
subjects, 90 percent reached the low benchmark in all three subjects. 
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Exhibit 1.18: Italy 

Profiles of High and Low Performance in Reading, Mathematics, and Science 


TIMSS & PIRLS 
2011 


^th 

Grade 


High International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 

Subjects I 

Percent of Students 

All Three Subjects 

18 (1.1) 

Reading 

46 (1.4) 

Mathematics 

28 (1.6) 

Science 

37 (1.6) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 


In At Least One Subject but Not All Three Science 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects I 

Percent of Students [ 

All Three Subjects 

90 (0.9) 

Reading 

98 (0.4) 

Mathematics 

93 (0.8) 

Science 

95 (1.0) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 


In At Least One Subject but Not All Three 


Science 


Reading 



Mathematics 
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Exhibit 1.19: Sweden 

Profiles of High and Low Performance in Reading, Mathematics, and Science 


TIMSS & PIRLS 
2011 


^th 

Grade 


High International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 

Subjects 

Percent of Students 1 

All Three Subjects 

18 (1.1) 

Reading 

47 (1.6) 

Mathematics 

25 (1.2) 

Science 

44 (1.5) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 


In At Least One Subject but Not All Three Science 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects 

Percent of Students 1 

All Three Subjects 

91 (0.7) 

Reading 

98 (0.3) 

Mathematics 

93 (0.6) 

Science 

95 (0.5) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 

I In At Least One Subject but Not All Three . 

Science 


Reading 



Mathematics 
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Exhibit 1.20: Austria 

Profiles of High and Low Performance in Reading, Mathematics, and Science 


TIMSS & PIRLS 
2011 


^th 

Grade 


High International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 

Subjects 1 

Percent of Students 1 

All Three Subjects 

18 (1.2) 

Reading 

39 (1.5) 

Mathematics 

26 (1.5) 

Science 

42 (1.7) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 



Science 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects 

Percent of Students j 

All Three Subjects 

92 (0.8) 

Reading 

97 (0.4) 

Mathematics 

95 (0.7) 

Science 

96 (0.6) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 


In At Least One Subject but Not All Three 


Science 


Reading 



Mathematics 
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Exhibit 1.21: Romania 

Profiles of High and Low Performance in Reading, Mathematics, and Science 


TIMSS & PIRLS 
2011 


^th 

Grade 


High International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 

Subjects 1 

Percent of Students 1 

All Three Subjects 

17 (1.2) 

Reading 

32 (1.6) 

Mathematics 

28 (1.7) 

Science 

37 (2.2) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 


In At Least One Subject but Not All Three Science 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects 1 

Percent of Students 1 

All Three Subjects 

73 (2.1) 

Reading 

86 (1.5) 

Mathematics 

79 (2.0) 

Science 

84 (1.8) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 


In At Least One Subject but Not All Three 


Science 


Reading 



Mathematics 
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Exhibit 1.22: Croatia 

Profiles of High and Low Performance in Reading , Mathematics, and Science 


TIMSS & PIRLS 
2011 


4 th 

Grade 


High International Benchmarks 


Students Reaching the TIMSS and PIRLS 2011 
High International Benchmarks 

Subjects 1 

Percent of Students 1 

All Three Subjects 

13 (0.7) 

Reading 

54 (1.3) 

Mathematics 

19 (1.0) 

Science 

30 (1.1) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 


In At Least One Subject but Not All Three Science 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 2011 
Low International Benchmarks 

Subjects 1 

Percent of Students J 

All Three Subjects 

90 (1.0) 

Reading 

99 (0.2) 

Mathematics 

91 (0.9) 

Science 

96 (0.5) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 

I In At Least One Subject but Not All Three . 

™ Science 


Reading 



Mathematics 
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POLAND There also was considerable variation in achievement in Poland. In 
reading, 39 percent reached the high benchmark and 29 percent did in science, 
while only 17 percent reached that level in mathematics. Twelve percent reached 
the high benchmark in all three subjects, and 83 percent reached the low level 
in all three subjects. The relative weakness in mathematics also was emerging 
at the low benchmark. While 95 and 91 percent of the students reached the low 
level in reading and science, respectively, 87 percent did in mathematics. 

SPAIN Similar to the pattern in Croatia and Poland, the Spanish students also 
showed a relative weakness in mathematics. Similar percentages of students 
reached the high benchmark in reading (30%) and in science (28%), while only 
17 percent did in mathematics. Nine percent reached the high benchmark in all 
three subjects, and 82 percent reached the low benchmark in all three subjects. 
The relative weakness in mathematics was noticeable at the low benchmark, 
with 94 and 92 percent of the students reaching the low benchmark in reading 
and science, but 87 percent in mathematics. 

NORWAY Norway had relatively similar percentages of students reaching the 
high benchmark in reading (25%), mathematics (21%), and science (19%). 
Interestingly, only 8 percent of those were the same students reaching the high 
benchmark in all three subjects. Also, high percentages of students reached the 
low benchmarks, more than 90 percent in each of the subjects. However, again 
somewhat fewer (86%) reached the low benchmark in all three subjects. 

MALTA The Maltese fourth grade students showed a relative weakness in science. 
One-fourth of the students reached the high benchmark in mathematics 
and reading, but only 14 percent in science. Seven percent reached the high 
benchmark in all three subjects, and 64 percent reached the low benchmark in 
all three subjects. At the low benchmark, the percentages indicated a relative 
strength in mathematics (88%), compared to 78 percent in reading and 70 
percent in science. 

UNITED ARAB EMIRATES Fourteen percent of the students reached the high 
benchmarks in reading and science, and 12 percent did in mathematics. Six 
percent reached the high benchmark in all three subjects, and about half (48%) 
reached the low benchmark in all three subjects. Achievement also was similar 
across the three subjects at the Low Benchmark (61-64%). 
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Exhibit 1.23: Poland 

Profiles of High and Low Performance in Reading, Mathematics, and Science 


TIMSS & PIRLS 
2011 


^th 

Grade 


High International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 

Subjects 1 

Percent of Students 

All Three Subjects 

12 (0.8) 

Reading 

39 (1.2) 

Mathematics 

17 (1.0) 

Science 

29 (1.5) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 


In At Least One Subject but Not All Three Science 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects I 

Percent of Students [ 

All Three Subjects 

83 (1.0) 

Reading 

95 (0.5) 

Mathematics 

87 (0.9) 

Science 

91 (0.8) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 

I In At Least One Subject but Not All Three r . 

“ Science 


Reading 



Mathematics 
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Exhibit 1.24: Spain 

Profiles of High and Low Performance in Reading, Mathematics, and Science 


TIMSS & PIRLS 
2011 


^th 

Grade 


High International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 

Subjects 

Percent of Students 1 

All Three Subjects 

9 (0.8) 

Reading 

30 (1.7) 

Mathematics 

17 (1.1) 

Science 

28 (1.5) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 


In At Least One Subject but Not All Three Science 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects 

Percent of Students j 

All Three Subjects 

82 (1.4) 

Reading 

94 (0.9) 

Mathematics 

87 (1.2) 

Science 

92 (1.2) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 



Science 


Reading 



Mathematics 
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Exhibit 1.25: Norway 

Profiles of High and Low Performance in Reading, Mathematics, and Science 


TIMSS & PIRLS 
2011 


^th 

Grade 


High International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 

Subjects 

Percent of Students 1 

All Three Subjects 

8 (0.9) 

Reading 

25 (1.6) 

Mathematics 

21 (1.6) 

Science 

19 (1.3) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 


In At Least One Subject but Not All Three Science 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects 

Percent of Students i 

All Three Subjects 

86 (1.2) 

Reading 

95 (0.8) 

Mathematics 

91 (0.9) 

Science 

92 (0.9) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 


In At Least One Subject but Not All Three 


Science 


Reading 



Mathematics 
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Exhibit 1.26: Malta 

Profiles of High and Low Performance in Reading, Mathematics, and Science 


TIMSS & PIRLS 
2011 


^th 

Grade 


High International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 

Subjects 

Percent of Students | 

All Three Subjects 

7 (0.4) 

Reading 

24 (0.7) 

Mathematics 

25 (0.9) 

Science 

14 (0.6) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 

| In At Least One Subject but Not All Three 


Science 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects 

Percent of Students | 

All Three Subjects 

64 (0.9) 

Reading 

78 (0.7) 

Mathematics 

88 (0.7) 

Science 

70 (1.0) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 

I In At Least One Subject but Not All Three r . 

“ Science 


Reading 



Mathematics 
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Exhibit 1.27: United Arab Emirates 

Profiles of High and Low Performance in Reading, Mathematics, and Science 


TIMSS & PIRLS 
2011 


^th 

Grade 


High International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 

Subjects 1 

Percent of Students I 

All Three Subjects 

6 (0.4) 

Reading 

14 (0.6) 

Mathematics 

12 (0.5) 

Science 

14 (0.5) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 


In At Least One Subject but Not All Three Science 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects I 

Percent of Students j 

All Three Subjects 

48 (1.1) 

Reading 

64 (0.9) 

Mathematics 

64 (1.0) 

Science 

61 (1.1) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 


In At Least One Subject but Not All Three 


Science 


Reading 



Mathematics 



TIMSS & PIRLS 

International Study Center 

Lynch School of Education, Boston College 


PROFILES OF ACHIEVEMENT ACROSS READING, 
MATHEMATICS, AND SCIENCE AT THE FOURTH GRADE 

CHAPTER 1 49 


SOURCE: lEA's Trends in International Mathematics and Science Study and Progress in International Reading Literacy Study -TIMSS and PIRLS 201 1 










GEORGIA The Georgian students found the assessments difficult, but showed 
a relative strength in reading. While 21 percent of the fourth grade students 
reached the high benchmark in reading, only 12-13 percent did in mathematics 
and science. Similarly, 87 percent reached the low benchmark in reading, 
compared with 72 percent in mathematics and 75 percent in science. Five 
percent reached the high benchmark in all three subjects and 65 percent reached 
the low benchmark in all three subjects. 

IRAN In Iran, students showed a slight comparative weakness in mathematics. 
Sixteen percent of the students reached the high benchmark in science and 
13 percent in reading, compared to 9 percent in mathematics. Four percent 
reached the high benchmark in all three subjects, and 57 percent reached the 
low benchmark in all three subjects. The pattern of mathematics being a relative 
weakness was noticeable at the low benchmark, 76 and 72 percent reaching this 
level in reading and science, respectively, but 64 percent in mathematics. 

QATAR Similar percentages (10-12%) reached the High International Benchmark 
in each of the three subjects, and 4 percent reached the high benchmark in all 
three subjects. Forty percent reached the Low International Benchmark in all 
three subjects, with 60 percent reaching this level in reading, 55 percent in 
mathematics, and 50 percent in science. 

AZERBAIJAN Interestingly, students in Azerbaijan showed a relative strength in 
mathematics at the high benchmark, and relative strength in reading at the low 
benchmark. The percentages of students reaching the high benchmark were 
21 percent in mathematics, but only 13 percent in science and 9 percent in 
mathematics. Three percent reached the high benchmark in all three subjects, 
and 55 percent reached the low benchmark in all three subjects. However, 82 
percent of the students reached the low benchmark in reading, compared to 72 
percent in mathematics and 65 percent in science. 

SAUDI ARABIA Twelve percent of the students reached the high benchmark 
in science, 8 percent in reading, and 7 percent in mathematics. Two percent 
reached the high benchmark in all three subjects and 43 percent reached the 
low benchmark in all three subjects, with performance in reading and science 
(63-65%) at the low benchmark relatively stronger than in mathematics (55%). 
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Exhibit 1.28: Georgia 

Profiles of High and Low Performance in Reading, Mathematics, and Science 


TIMSS & PIRLS 
2011 


^th 

Grade 


High International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 

Subjects 1 

Percent of Students 1 

All Three Subjects 

5 (0.7) 

Reading 

21 (1.2) 

Mathematics 

12 (1.0) 

Science 

13 (1.2) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 


In At Least One Subject but Not All Three 


Science 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects 1 

Percent of Students 

All Three Subjects 

65 (1.6) 

Reading 

87 (1.4) 

Mathematics 

72 (1.7) 

Science 

75 (1.6) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 

I In At Least One Subject but Not All Three r . 

“ Science 


Reading 



Mathematics 
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Exhibit 1.29: Iran, Islamic Republic of 

Profiles of High and Low Performance in Reading, Mathematics, and Science 


TIMSS & PIRLS 
2011 


^th 

Grade 


High International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 

Subjects 1 

Percent of Students 

All Three Subjects 

4 (0.5) 

Reading 

13 (0.9) 

Mathematics 

9 (0.8) 

Science 

16 (1.1) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 



Science 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects 1 

Percent of Students | 

All Three Subjects 

57 (1.6) 

Reading 

76 (1.2) 

Mathematics 

64 (1.5) 

Science 

72 (1.5) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 

I In At Least One Subject but Not All Three . 

“ Science 


Reading 



Mathematics 
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Exhibit 1.30: Qatar 

Profiles of High and Low Performance in Reading , Mathematics, and Science 


TIMSS & PIRLS 
2011 


^th 

Grade 


High International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 

Subjects 1 

Percent of Students [ 

All Three Subjects 

4 (0.7) 

Reading 

12 (1.1) 

Mathematics 

10 (0.9) 

Science 

11 (1.0) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 

| In At Least One Subject but Not All Three 


Science 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 2011 
Low International Benchmarks 

Subjects 

Percent of Students j 

All Three Subjects 

40 (1.6) 

Reading 

60 (1.5) 

Mathematics 

55 (1.5) 

Science 

50 (1.5) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 


In At Least One Subject but Not All Three 


Science 


Reading 



Mathematics 



TIMSS & PIRLS 

International Study Center 

Lynch School of Education, Boston College 
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Exhibit 1.31: Azerbaijan 

Profiles of High and Low Performance in Reading, Mathematics, and Science 


TIMSS & PIRLS 
2011 


^th 

Grade 


High International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 

Subjects 1 

Percent of Students 1 

All Three Subjects 

3 (0.7) 

Reading 

9 (0.9) 

Mathematics 

21 (2.3) 

Science 

13 (1.7) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 

| In At Least One Subject but Not All Three 


Science 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects 

Percent of Students j 

All Three Subjects 

55 (2.3) 

Reading 

82 (1.6) 

Mathematics 

72 (1.9) 

Science 

65 (2.0) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 


In At Least One Subject but Not All Three 


Science 


Reading 



Mathematics 
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Exhibit 1.32: Saudi Arabia 

Profiles of High and Low Performance in Reading, Mathematics, and Science 


TIMSS & PIRLS 
2011 


^th 

Grade 


High International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 

Subjects I 

Percent of Students | 

All Three Subjects 

2 (0.7) 

Reading 

8 (1.0) 

Mathematics 

7 (1.2) 

Science 

12 (1.4) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 


In At Least One Subject but Not All Three Science 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects 1 

Percent of Students j 

All Three Subjects 

43 (1.8) 

Reading 

65 (1.8) 

Mathematics 

55 (1.8) 

Science 

63 (1.9) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 


In At Least One Subject but Not All Three 


Science 


Reading 



Mathematics 
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OMAN Performance was similar across the three subjects in Oman. Relatively 
small percentages (5-7%) of students reached the High International Benchmark 
in each of the three subjects, with 1 percent reaching the high level in all three 
subjects. Thirty percent reached the low benchmark in all three subjects, 
although nearly half (45-48%) reached the low level in each subject. 

MOROCCO Performance was also similar across the three subjects in Morocco. 
Few students (1-2%) reached the high benchmarks in reading, mathematics, 
and science, but about one-fourth reached the low benchmark in mathematics, 
21 percent in reading, and 15 percent in science. 

Countries at the Sixth Grade 

BOTSWANA At the high benchmark, performance in Botswana was similar across 
the three subjects. Nine percent of the sixth grade students reached the high 
benchmark in reading, while 7 percent did so in mathematics and science. 
Three percent reached the high benchmark in all three subjects. At the Low 
International Benchmark, students showed a relative weakness in science. 
Thirty-seven percent reached the low benchmark in all three subjects, with 60 
percent reaching that level in reading and 56 percent in mathematics, but only 
43 percent in science. 

HONDURAS In Honduras, students showed a relative weakness in mathematics 
at both the high and the low benchmark. At the high benchmark, 1 1 percent of 
the sixth grade students reached the benchmark in reading and 8 percent did 
so in science, but only 3 percent reached this level in mathematics. Because the 
students performing well in mathematics mostly also did well in the other two 
subjects, 2 percent reached the high benchmark in all three subjects. Forty- 
three percent reached the low benchmark in all three subjects, with considerable 
variation across the subjects. Approximately three-fourths of the students (74%) 
reached that level in reading, two-thirds (66%) in science, and half (49%) in 
mathematics. 

Benchmarking Participants 

QUEBEC, CANADA The students in Quebec showed relative weakness in science, 
with 43 percent of students reaching the High International Benchmark in 
reading and 40 percent in mathematics, compared to 29 percent in science. 
Seventeen percent reached the high benchmark in all three subject and 
essentially all students (95%) reached the low benchmark in all three subjects. 
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Exhibit 1.33: Oman 

Profiles of High and Low Performance in Reading, Mathematics, and Science 


TIMSS & PIRLS 
2011 


^th 

Grade 


High International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 

Subjects I 

Percent of Students 

All Three Subjects 

1 (0.2) 

Reading 

5 (0.4) 

Mathematics 

5 (0.3) 

Science 

7 (0.7) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 


In At Least One Subject but Not All Three Science 


Reading 



Mathematics 


Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects I 

Percent of Students j 

All Three Subjects 

30 (1.0) 

Reading 

48 (1.2) 

Mathematics 

46 (1.2) 

Science 

45 (1.4) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 

I In At Least One Subject but Not All Three . 

“ Science 


Reading 



Mathematics 
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DUBAI, UNITED ARAB EMIRATES Performance was very similar across the 
three subjects. Approximately one-fourth of the students reached the High 
International Benchmark in each subject — 26 percent in reading, 23 percent in 
science, and 22 percent in mathematics. Twelve percent of the students reached 
the high benchmark in all three subjects, and 63 percent reached the low 
benchmark in all three subjects. Approximately three-fourths of the students 
reached the Low International Benchmark in each subject — 75 percent in both 
reading and mathematics, and 72 percent in science. 

ABU DHABI, UNITED ARAB EMIRATES Performance in Abu Dhabi also was very 
similar across the three subjects. Ten percent of the students reached the high 
benchmark in reading and science, and 8 percent did in mathematics. Three 
percent reached the high benchmark in all three subjects and 43 percent 
reached the low benchmark in all three subjects. At the low benchmark, 60 
percent reached this level in reading, 58 percent in mathematics, and 55 percent 
in science. 

Summary 

Students performing at the High International Benchmarks in all three subjects 
are very accomplished fourth grade students — able to read relatively complex 
materials with in-depth understanding, solve a variety of mathematics problems, 
and show familiarity with a range of scientific information. These students have 
developed a solid basis for further learning and are well positioned to take 
advantage of future educational opportunities. However, the TIMSS and PIRLS 
2011 data provide evidence that it is a very challenging task to educate students 
to the level of the high benchmarks at the fourth grade. Only Singapore had 
more than half its students reach the high benchmarks in all three subjects, and 
only two more countries, Chinese Taipei and Finland, had at least half their 
fourth grade students reach the high benchmark in each subject separately. 
Chinese Taipei had 40 percent of its students reach the high benchmark in all 
three subjects and Finland had 39 percent, as did Hong Kong SAR. The Russian 
Federation had 35 percent reach the high benchmark in all three subjects, and 
the remaining participants had less than 30 percent. 

More than half the countries, however, were successful in educating 90 
percent of more of their students to the Low International Benchmark in all 
three subjects. These students showed that they can read and comprehend 
facts, read a variety of simple graphs and tables, know simple mathematics 
(such as adding, subtracting, and basic geometric figures), and know science 
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Exhibit 1.34: Morocco 


TIMSS & PIRLS 
2011 


^th 

Grade 


Profiles of High and Low Performance in Reading, Mathematics, and Science 


Sixth Grade Participant 

Students Reaching the TIMSS and PIRLS 201 1 

High International Benchmarks 

Subjects | Percent of Students 


All Three Subjects 0 (0.1) 

Reading 1 (0.2) 

Mathematics 2 (0.6) 

Science 1 (0.4) 

( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 


In At Least One Subject but Not All Three Science 


High International Benchmarks 

Reading 



Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects 1 

Percent of Students j 

All Three Subjects 

8 (0.9) 

Reading 

21 (1.3) 

Mathematics 

26 (1.5) 

Science 

15 (1.0) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 

| In At Least One Subject but Not All Three 


Science 


Reading 



Mathematics 
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Exhibit 1.35: Botswana 

Profiles of High and Low Performance in Reading , Mathematics, and Science 


TIMSS & PIRLS 
2011 


^th 

Grade 


Sixth Grade Participant 

Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 


All Three Subjects 3(0.7) 

Reading 9 (1.3) 

Mathematics 7 (1.0) 

Science 7 (1.1) 


( ) Standard errors appear in parenthesis. 


Percent of Students 


Subjects 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 


In At Least One Subject but Not All Three Sci6nC6 


High International Benchmarks 

Reading 



Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects I 

Percent of Students ! 

All Three Subjects 

37 (1.8) 

Reading 

56 (1.8) 

Mathematics 

60 (1.6) 

Science 

43 (1.8) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 

| In At Least One Subject but Not All Three 


Science 


Reading 



Mathematics 
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Exhibit 1.36: Honduras 


TIMSS & PIRLS 
2011 


^th 

Grade 


Profiles of High and Low Performance in Reading, Mathematics, and Science 


Sixth Grade Participant 

Students Reaching the TIMSS and PIRLS 2011 
High International Benchmarks 


All Three Subjects 2(0.6) 

Reading 11 (1.5) 

Mathematics 3 (0.9) 

Science 8 (1.4) 


( ) Standard errors appear in parenthesis. 


Percent of Students 


Subjects 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 


In At Least One Subject but Not All Three Science 


High International Benchmarks 

Reading 



Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 2011 
Low International Benchmarks 

Subjects I 

Percent of Students [ 

All Three Subjects 

43 (2.5) 

Reading 

74 (2.2) 

Mathematics 

49 (2.5) 

Science 

66 (2.5) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 


In At Least One Subject but Not All Three 


Science 


Reading 



Mathematics 
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Exhibit 1.37: Quebec, Canada 


TIMSS & PIRLS 
2011 


^th 

Grade 


Profiles of High and Low Performance in Reading, Mathematics, and Science 


Benchmarking Participant 

Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 


All Three Subjects 17 (1.2) 

Reading 43 (1.8) 

Mathematics 40 (1.7) 

Science 29 (1.5) 


( ) Standard errors appear in parenthesis. 


Percent of Students 


Subjects 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 


In At Least One Subject but Not All Three Science 


High International Benchmarks 


Reading 



Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects 1 

Percent of Students | 

All Three Subjects 

95 (0.5) 

Reading 

98 (0.3) 

Mathematics 

99 (0.3) 

Science 

97 (0.4) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 

| In At Least One Subject but Not All Three 


Science 


Reading 



Mathematics 
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Exhibit 1.38: Dubai, UAE 


TIMSS & PIRLS 
2011 


^th 

Grade 


Profiles of High and Low Performance in Reading, Mathematics, and Science 


Benchmarking Participant 

Students Reaching the TIMSS and PIRLS2011 
High International Benchmarks 
Subjects | Percent of Students 


All Three Subjects 12(0.7) 

Reading 26 (0.8) 

Mathematics 22 (0.8) 

Science 23 (0.9) 

( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 


All Three Subjects 


In At Least One Subject but Not All Three ScienC6 


High International Benchmarks 


Reading 



Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 2011 
Low International Benchmarks 

Subjects 1 

Percent of Students 1 

All Three Subjects 

63 (1.2) 

Reading 

75 (0.8) 

Mathematics 

75 (0.8) 

Science 

72 (1.1) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 


In At Least One Subject but Not All Three 


Science 


Reading 



Mathematics 
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Exhibit 1.39: Abu Dhabi, UAE 


TIMSS & PIRLS 
2011 


^th 

Grade 


Profiles of High and Low Performance in Reading, Mathematics, and Science 


Benchmarking Participant 

Students Reaching the TIMSS and PIRLS 201 1 
High International Benchmarks 
Subjects | Percent of Students 


All Three Subjects 3 (0.6) 

Reading 10 (1.2) 

Mathematics 8 (1.0) 

Science 10 (1.0) 

( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 

| In At Least One Subject but Not All Three 


Science 


High International Benchmarks 

Reading 



Low International Benchmarks 


Students Reaching the TIMSS and PIRLS 201 1 
Low International Benchmarks 

Subjects 1 

Percent of Students 1 

All Three Subjects 

43 (2.1) 

Reading 

60 (1.9) 

Mathematics 

58 (2.1) 

Science 

55 (2.2) 


( ) Standard errors appear in parenthesis. 


Percent of Students Reaching the High 
TIMSS & PIRLS International Benchmarks 

All Three Subjects 

I In At Least One Subject but Not All Three . 

“ Science 


Reading 



Mathematics 
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facts about health, ecosystems, and animals. Although these students have 
lower achievement than those at the high level, they do have a well-rounded 
foundation in core concepts and skills that provides a good basis for further 
learning. In comparison, students who have not learned the basic fundamentals 
of reading, mathematics, and science by the end of their fourth year of schooling 
may be at some risk for future academic success. 

Interestingly, most countries were more successful in educating their 
students in one or two of the subjects than in the others, especially when it 
comes to educating substantial percentages of students to high levels. For 
example, among the five countries with the highest percentages of students 
reaching the High International Benchmark, the three East Asian countries had 
a particular strength in mathematics — Singapore, Chinese Taipei, and Hong 
Kong SAR. In contrast, Finland had relative weakness in mathematics compared 
to its relative strengths in reading and science. The Russian Federation showed a 
particular strength in reading. Relatively few countries had similar percentages 
of students reach the benchmarks across all three subjects. 
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Chapter 2 


The Impact of Reading Ability on 
TIMSS Mathematics and Science 
Achievement at the Fourth Grade: 

An Analysis by Item Reading Demands 

Ina V.S. Mullis, Michael 0. Martin, and Pierre Foy 

Boston College 

Introduction 

In the past several decades, schools have seen increasing 
integration across subject areas in teaching and learning, 
including greater emphasis on reading within subject areas. 
Today mathematics and science curricula around the world, as 
well as standards for proficiency in these subjects, commonly 
include reading and communication skills, and the TIMSS 201 1 
Assessment Frameworks (Mullis, Martin, Ruddock, O’Sullivan, 
& Preuschoff, 2009) reflect this situation. At the fourth grade 
in mathematics, for example, topics in the TIMSS framework 
content domains of number, measurement, and data display 
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specify that students should be able to solve routine and non-routine problems 
set in everyday contexts. Understanding the description of the everyday 
situations for these types of problems necessarily involves reading. Also, the 
data display area is based on “reading and interpreting” tables, pictographs, bar 
graphs, and pie charts as well as creating such data displays. Similarly, the science 
framework requires students to comprehend descriptions of experiments and 
investigations as well as to read and interpret a variety of models and diagrams 
of science systems and phenomena. 

Given that reading has been incorporated into the mathematics and science 
assessment frameworks, the TIMSS 2011 fourth grade achievement items 
encompass a range of reading comprehension demands. Of course, the reading 
demands vary across items, from quite minimal, as in items requiring students 
to complete a calculation, to somewhat more substantial, as in items requiring 
students to understand a description of a science experiment or phenomenon 
and then apply their knowledge or explain their reasoning. Additionally, it 
should be emphasized that all TIMSS items undergo extensive review for clarity, 
straightforward vocabulary, and syntax, and that any extraneous or irrelevant 
information is removed. Developing clearly written items that are equally 
accessible to all students is fundamental to all item development, but especially 
pertinent to TIMSS which needs to be translated into 30 or so languages. 
Multiple reviews by representatives of the participating countries ensure that 
the TIMSS item development approach prioritizes clarity and brevity, such 
that none of the TIMSS fourth grade items involve reading of any length or 
complexity (although some do at the eighth grade). 

Still, fourth grade students are likely to be at a disadvantage in learning 
mathematics and science as well as demonstrating high performance on the 
TIMSS assessment if they lack reading skills. The availability of PIRLS data 
on reading achievement at the fourth grade provides an ideal opportunity to 
investigate the relationship between reading ability and the reading demands 
of the TIMSS mathematics and science assessment items. This study capitalized 
on the unique availability of PIRLS and TIMSS achievement scores for the same 
fourth grade students across reading, mathematics, and science for 34 countries 
and three benchmarking participants to examine the following overarching 
question: 


How does reading ability impact TIMSS mathematics and 
science achievement at the fourth grade? 
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Overview of Study 

The basic approach used in the analysis presented in this chapter was to examine, 
for each participating country and benchmarking entity, the relationship 
between fourth grade students’ reading ability as measured by PIRLS and their 
performance on TIMSS items with increasing levels of reading demands. 

The hypotheses were as follows: 

1. Students with high reading ability would not be impacted by the level 
of reading demand in the items. That is, the best readers would score 
similarly on TIMSS items regardless of the degree of reading demands. 

2. Students with lower reading ability would perform relatively better on 
items with less reading. That is, poorer readers would score better on 
the items with the lowest reading demands than on the items with the 
highest reading demands. 

The study was conducted separately for mathematics and for science, with 
the initial steps involving sorting the 175 TIMSS fourth grade mathematics items 
and 168 science items according to degree of reading demands. To maintain 
robustness of measurement while at the same time preserving differentiation, 
each set of fourth grade TIMSS items (mathematics and science) was separated 
into three relatively equal sized categories from relatively low to relatively high 
reading demands (low, medium, and high). Students’ mathematics and science 
achievement was examined for each of the groups of mathematics and science 
items categorized as having low, medium, and high reading demands, for 
students at three different levels of reading ability on PIRLS 2011 (upper, middle, 
and lower terciles). The hypotheses were supported in general, but more so in 
mathematics than in science, and more so in some countries than in others. 
The results differed across countries, and sometimes between mathematics and 
science within countries. 

Categorizing the TIMSS Fourth Grade Mathematics and 
Science Items According to Reading Demands 

It was fundamental to this study to be able rank the TIMSS fourth grade 
mathematics and science items by level of reading demand in a manner 
that would be reliable and appropriate for further analysis. To have enough 
categories to discriminate between items, but not too many categories such that 
distinguishing among them would become extremely difficult, it was decided to 
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have three categories of reading demand: low, medium, and high. Furthermore, 
it was very important to be able to document that the lowest category included 
items with lower reading demands than the medium category and, in particular, 
the highest category Thus, the process of separating the items into the three 
categories of low, medium, and high reading demands involved several phases, 
including a review of the literature about the factors influencing reading 
demand, a holistic evaluation of the items according to selected indicators 
of reading demand, coding each item according to the reading demand 
indicators, and validating the holistic item categorizations through discriminant 
function analysis. 

Holistic Evaluation of the Level of Reading Demands in the 
TIMSS 201 1 Fourth Grade Items 

As a first step toward holistically rating the TIMSS fourth grade items according 
to level of reading demand, the TIMSS & PIRLS International Study Center staff 
conducted a detailed review of the literature concerning dimensions of reading 
difficulty in the context of evaluating the reading demands presented by the 
TIMSS 2011 fourth grade mathematics and science items. 

Settling on the best set of indicators to capture the reading demand of 
the TIMSS items was somewhat challenging, because much of the research on 
factors influencing reading difficulty is based on continuous text of some length, 
whereas the TIMSS fourth grade items are short. Also, the preponderance of 
research about reading difficulty in test items highlights how difficulty can be 
reduced by using clearer, less complicated language; however TIMSS already 
makes every effort to avoid unnecessary reading and the language used is 
no more complex than needs to be to frame the question (and responses for 
multiple-choice items). Additionally, the reading demand indicators used for 
this study needed to be applicable across the many languages of the TIMSS 
countries, which imposed further operational and practical constraints. When 
considering which of the many factors that influence reading difficulty could be 
used as indicators of reading demand, the following criteria were kept in mind: 

♦ Appropriateness for the TIMSS fourth grade items; 

♦ Generalizability across languages; 

♦ Likelihood of being applied reliably; and 

♦ Feasibility within resource and time constraints. 
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After a detailed discussion, staff articulated a set of indicators that appeared 
most applicable to evaluating the reading demands of the TIMSS fourth grade 
items. These included: 

1. Number of Words — The number of words one must read is a basic 
feature of reading difficulty included in many well-known readability 
formulae (e.g., Dale-Chall and Flesch-Kincaid). While it was recognized 
that the number of words varies across languages, it was assumed that 
the items with more words in English would also have more words in 
other languages; therefore the relative reading demands across items 
would be maintained. 

2. Vocabulary — A unique feature of reading in the mathematics and 
science context is that there are specialized vocabularies one must 
know for complete comprehension, but the use of particular vocabulary 
terms can contribute to reading demand (Adams, 2003; Bernardo, 2005; 
Justenson & Katz, 1995; Kane, Bryne, & Hater, 1974). Although there 

is some debate, both mathematics and science generally are regarded 
as having specialized languages with their own technical vocabularies, 
including everyday language that has specific meaning when used in 
the mathematics or science context (e.g., “difference” and “more” in 
mathematics; and “stay alive” instead of “survive” in science). 

3. Symbolic Language — Similar to the specialized vocabulary component, 
understanding symbolic language requires reading skills that are 
particularly important in the mathematics and science context 
(Matteson, 2006). These can include numerals (e.g., 3, 5, 40) as well as 
other symbols and abbreviations (e.g., +, =, cm). 

4. Visual Displays — The TIMSS achievement items contain a range of 
visual displays that students need to interact with to varying degrees in 
order to successfully complete the items. The complexity or density of 
a visual display impacts reading difficulty (Matteson, 2006; Mosenthal 
& Kirsch, 1998). Visual displays included the following: 1) pictorial 
representations of real world things, 2) geometric shapes and figures, 3) 
models and diagrams, 4) tables, and 5) graphs. 
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The above indicators of reading difficulty were used to holistically rate 
the TIMSS items according to their reading demands. Taking the specified 
components into account, ten members of the TIMSS & PIRLS International 
Study Center with backgrounds in measurement, reading, mathematics, and 
science used a holistic approach to evaluate the reading demands required by 
each item as low, medium, or high. According to the holistic scoring approach, 
the categorization was based on the overall impression of the reading difficulty 
of the item, with the proviso to assign about the same number of items to each 
category to ensure stability in the analyses. After independently rating each of 
the items, the entire team met to reconcile results and reach group consensus 
on the holistic rating of each item as low, medium, or high. 

Empirical Data About the Reading Difficulty Factors 
Present in Each Item 

The next phase of the study involved validating the holistic ratings, by coding 
each of the items according to the four dimensions of reading difficulty. The 
TIMSS & PIRLS International Study Center staff developed a draff coding guide 
to identify and quantify the difficulty factors present in each of the TIMSS 
fourth grade mathematics and science items. This draff underwent a series of 
internal reviews. Then, at the June 2012 meeting of TIMSS and PIRLS 2011 
National Research Coordinators (NRCs) in Singapore, NRCs were led through 
the draff coding guide, and they suggested ways in which the coding guide could 
be further improved. 

Most importantly, there was a thorough debate among the NRCs about the 
intersection of reading and mathematics, particularly in the areas of symbolic 
language and geometric shapes. That is, when students are asked simply to 
solve an equation for “X,” are they reading the language of mathematics or only 
“doing mathematics” without any reading? Similarly, when asked to analyze the 
attributes of a triangle, are students engaging in an activity similar to reading 
a diagram, or is that only “doing mathematics?” Based on the literature review 
and the discussion, the TIMSS & PIRLS International Study Center decided 
to code these aspects of reading difficulty with separate codes so that analyses 
could be done with or without these aspects should researchers be interested. 
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The major indicators of reading demand in the final coding guide applied 
to each TIMSS fourth grade item were as follows: 


♦ The number of words 
(anywhere in the item, 
including titles of 
graphics and labels); 

♦ The number of 
different symbols (e.g., 
numerals, operators); 

♦ The number of 
different specialized 
vocabulary words; and 

♦ The total number of 
elements (density) 
in the visual displays 
(e.g., diagrams, graphs, 
tables). 


Exhibit 2.1 : Mathematics Item Coding Example 



Exhibit 2.2: Science Item Coding Example 


Science Item Coding Example 


Stephanie has a balance and four cubes (1, 2, 3, 4). The cubes are made of 
different materials. 

She puts two cubes at a time on the balance and observes the following results. 




The coding guide was 
implemented to document 
the reading demands of each 
of the TIMSS fourth grade 
mathematics and science 
items (see Technical Appendix 
A: Quantifying the Reading 
Demands of the TIMSS 2011 
Fourth Grade Mathematics 
and Science Items). In order 
to ensure that all dimensions 
of difficulty were correctly 
coded, each item was coded 

independently by two TIMSS or PIRLS senior staff members, who then 
reconciled any discrepancies. Exhibits 2.1 and 2.2 provide examples of the 
detailed coding applied to each item. 



Number of 
Words: 74 


What can she conclude about the weight of cube 2? 

(A) It is heavier than cubes 1, 3, and 4. 

(b) It is heavier than cube 1 but lighter than cubes 3 and 4. 
© It is heavier than cube 3 but lighter than cubes 1 and 4. 
@ It is heavier than cube 4 but lighter than cubes 1 and 3. 


Symbolic Language: 4 different 
symbols (1,2, 3, 4) 

Vocabulary: 2 different terms 
(balance, cube) 

Visual Display: 3 models, each with 
a density of 3 (1 balance, 2 cubes) 
and a "necessary" level of interaction 
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Discriminant Function Analysis 

Finally, in order to validate the holistic categorizations of the items according 
to low, medium, and high reading demands, the data from coding the reading 
demands were used to conduct a discriminant function analysis (DFA). Group 
membership according to the holistic ratings was predicted using the four 
reading demand indicators: number of words, number of different symbols, 
number of different specialized vocabulary terms, and number and density of 
visual displays. Exhibit 2.3 presents the results of the DFAs for mathematics 
and science. For both subjects, the first discriminant function was sufficient 
to discriminate between the item groups, and the number of words was the 
indicator that loaded most heavily on this function. 


Exhibit 2.3: Discriminant Function Analysis Results 





1 


1 

Loading of Reading Demand 
Indicators on Discriminant Functions 

Function 

Reading Demand Indicators 

1 


2 


Total Number 
of Words 

.897 

_ 

-.137 


Sum of Visual Display 
Density and 
Interaction Values 

.327 


.360 


Number of 
Unique Symbols 

.203 


-.376 


Number of Unique 
Technical Words 

.016 


.806 







1 


1 

Loading of Reading Demand 
Indicators on Discriminant Functions 

Function 

Reading Demand Indicators 

1 

L 

2 


Total Number 
of Words 

.889 


-.407 


Sum ofVisual Display 
Density and 
Interaction Values 

.489 


.807 


Number of 
Unique Symbols 

.200 


.333 


Number of Unique 
Technical Words 

.060 


.282 



Exhibit 2.4 presents the DFA classification results, which show that the 
reading demand indicators were effective in recovering the low, medium, and 
high holistic categorizations. The predicted categories largely matched the 
holistic categories, with agreement on 82 percent of the items for mathematics 
and 77 percent of the science items. 
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Exhibit 2.4: Discriminant Function Analysis Classification Results — Confirmation 
of Item Classification by Reading Demands Based on Holistic 
Evaluation and Predicted by Discriminant Function Analysis* 


Mathematics 


Reading Demand Group Predicted by DFA 




Low 

|f|J Medium |||| 

High |i 

Reading 

Demand Group — 

Low 

53 

9 

0 

Holistic 

Evaluation 

Medium 

11 

46 

2 


High 

0 

9 

45 


Science 


Reading 

Demand Group — 
Holistic 
Evaluation 


Reading Demand Group Predicted by DFA 


Low 


41 


Medium 


12 


Medium 


56 


High 


18 


33 


Shaded cells show number of holistically evaluated items in agreement with DFA. 


Characteristics of Reading Demands in the 
TIMSS 201 1 Fourth Grade Items 

As explained earlier, the TIMSS 2011 fourth grade mathematics and science 
items typically do not have heavy reading demands, although some can be 
somewhat challenging. Exhibit 2.5 shows the characteristics of the items in 
terms of the four indicators of reading difficulty used in this study. On average, 
the total number of words (including all words appearing anywhere in the 
item) was relatively low, especially for mathematics, which included some 
items simply asking for computation. The average number of words in the 
mathematics items was 25 with a maximum of 84 words, and the average for 
the science items was 41 words with a maximum of 151 words. As would be 
anticipated, the mathematics items had more symbolic language (e.g., numerals 
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and operators) than the science items, but still averaged only 5 unique symbols 
per item. Because the language in these items was intended to be at the fourth 
grade level or lower, occurrences of specialized mathematics and science terms 
were low (2 to 3 terms per item on average). Finally, the number of visual 
displays refers to the total density or number of elements in the visual display or 
displays in the item. Because the TIMSS fourth grade mathematics framework 
includes geometric shapes as well as data displays (e.g., tables and graphs), 
the mathematics items more often included visual displays with a number of 
elements (8 on average) than the science items (3 on average). 

Exhibit 2.5: Indicators of Reading Difficulty for the TIMSS 2011 Fourth Grade Items 



* Includes all words appearing anywhere in the item — stem, question, response categories, and visual displays 
(e.g., exhibit titles, labels) 

** Includes numerals, signs of operations, units (e.g., abbreviations such as cm), variables (e.g., X), and labels 
(e.g., A for angle A). 

*** Includes all elements in the visual display(s). 


Exhibits 2.6a through 2.6f present examples of TIMSS 2011 mathematics 
and science items with low, medium, and high reading demands, respectively. 
The items in the low reading demands category typically were very 
straightforward, including mathematics computation items with hardly any 
words, short constructed response items where the question asked for the 
answer in a word or phrase, and basic multiple choice items with a question 
and short options. The items in the medium category had more words, and 
also often had diagrams or geometric figures especially for mathematics. In 
comparison, the items in the high category usually had both more words and 
more complex visual displays. 
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Exhibit 2.6: Examples of TIMSS Fourth Grade Mathematics and Science Items 
Categorized as Having Low, Medium, and High Reading Demands 

Exhibit 2.6a: Example TIMSS Fourth Grade Mathematics Item Categorized as Having 
Low Reading Demands 


TIMSS &PIRLS 
2011 


23 x 19 = 
Answer: _ 


LO 

O 


Exhibit 2.6b: Example TIMSS Fourth Grade Science Item Categorized as Having 
Low Reading Demands 




m 

o 

cn 


Which of these animals has a young form that looks the most like the adult 
form? 

® moth 
® human 
© frog 
@ butterfly 
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Grade 



Exhibit 2.6: 


Examples of TIMSS Fourth Grade Mathematics and Science Items TIMSS fr PIRLS 

Categorized as Having Low, Medium, and High Reading Demands (Continued) 2011 


^th 

Grade 


Exhibit 2.6c: Example TIMSS Fourth Grade Mathematics Item Categorized as Having 
Medium Reading Demands 



Which of the following shows the position of the shape above after a half turn or 1 
180° rotation? 1 
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Exhibit 2.6: Examples of TIMSS Fourth Grade Mathematics and Science Items TIMSS &PIRLS 

Categorized as Having Low, Medium, and High Reading Demands (Continued) 2011 

Exhibit 2.6d: Example TIMSS Fourth Grade Science Item Categorized as Having 
Medium Reading Demands 

o 

Some of the materials below will burn and some will not. | 

Put an X in the box next to the materials that will burn. ■? 

ro 
cn 
to 

(You may put an X in more than one box.) p 


j water 
" j wood 
j j sand 
| j gasoline 

□ air 
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Exhibit 2.6: 


Examples of TIMSS Fourth Grade Mathematics and Science Items TIMSS fr PIRLS 

Categorized as Having Low, Medium, and High Reading Demands (Continued) 2011 


^th 

Grade 


Exhibit 2.6e: Example TIMSS Fourth Grade Mathematics Item Categorized as Having 
High Reading Demands 



The graph shows the number of students at each grade in the Pine School. 


c 

CD 

T3 

3 


CD 

-Q 


Pine School 



Grade 


In the Pine School there is room in each grade for 30 students. 


How 

many more students could be in the school? 

ai 

JZ 

TO 

2 

® 

20 
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C 

o 

ro 

c 

(D 

25 
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Exhibit 2.6: 


Examples of TIMSS Fourth Grade Mathematics and Science Items TIMSS &PIRLS 

Categorized as Having Low, Medium, and High Reading Demands (Continued) 2011 


Exhibit 2.6f: Example TIMSS Fourth Grade Science Item Categorized as Having 
High Reading Demands 





Answer the following questions using the animals shown above. Write the name 
for the correct animal in the spaces below. 

Which animal has an internal skeleton and produces milk for its young? 


Which animal has an external skeleton and three pairs of legs? 


Which animal has a soft body and no skeleton? 


to 
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Interaction Between the Levels of Reading Demands and 
the TIMSS 201 1 Content and Cognitive Domains at the 
Fourth Grade 

It should be recognized that the nature of the content and cognitive domains in 
TIMSS frameworks can heavily influence the reading demands required by the 
items. Exhibit 2.7 shows the content and cognitive domains for the TIMSS 2011 
mathematics and science items at the fourth grade. Of these two dimensions of 
the mathematics and science assessments, the content domains describe in some 
detail the major content to be assessed and the cognitive domains describe the 
thinking skills the students should be using within the content domains. The 
TIMSS 2011 mathematics and science assessments each encompassed three 
content domains and three cognitive domains. The cognitive domains were 
the same for mathematics and science and at the fourth grade and they had the 
same amount of emphasis. 


Exhibit 2.7: TIMSS 201 1 Content and Cognitive Domains at the Fourth Grade 
Percentages Devoted to Each Domain 


Mathematics 

■ 

Science 

■ 

Mathematics 
and Science 


Content 

Domains 


Percentage 

of 

Assessment 


Content 

Domains 


Percentage 

of 

Assessment 


Cognitive 

Domains 


Percentage 

of 

Assessment 

Number 

50% 

Life Science 

45% 

Knowing 

40% 

Geometric Shapes 
and Measures 

35% 

Physical Science 

35% 

Applying 

40% 

Data Display 

15% 

Earth Science 

20% 

Reasoning 

20% 


Exhibit 2.8 presents the distributions by content domain of the TIMSS 2011 
mathematics and science fourth grade items categorized as having low, medium, 
and high reading demands. For mathematics, there is a clear interaction 
between the content domain and the reading demands of the items. Half of 
the assessment is devoted to assessing the number domain, with 50 percent 
of the number items being classified as having low reading demands because 
many ask only for computation or familiarity with basic number concepts. 
The geometry and measurement items (about a third of the assessment) often 
were categorized as medium (49%), because the framework calls for assessing a 
variety of understandings related to points, lines, and angles as well as two- and 
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three-dimensional shapes. Therefore, the typical item in this content domain 
includes a figure with an associated question, with some items being very 
straightforward and others more complicated, but primarily the items were 
in-between. Finally, although only a small part of the assessment is devoted to 
data display, most of these items (85%) were categorized as having high reading 
demands because, consistent with the framework topics, these items typically 
involved reading and interpreting data from relatively dense visual displays, 
including tables, pictographs, bar graphs, and pie charts. 


Exhibit 2.8: TIMSS Fourth Grade Mathematics and Science Items by Content Domains and 
Low, Medium, and High Reading Demands 


Mathematics Science 

Content Domains Content Domains 


Reading 

Demands 


Number 


Geometric 

Shapes 

and 

Measures 


Data 

Display 


Total 

Math 


Life 

Science 


Physical 

Science 


Earth 

Science 


Total 

Science 

Low 

50% 


28% 


4% 


35% 


36% 


26% 


30% 


32% 

Medium 

30% 


49% 


12% 


34% 


35% 


33% 


55% 


38% 

High 

20% 


23% 


85% 


31% 


28% 


41% 


15% 


30% 

Total 

88 


61 


26 


175 


74 


61 


33 


168 


Interestingly, the degree of difficulty of the TIMSS mathematics items 
across content domains may be unexpected in light of the relative levels 
of reading demands. As shown in the report containing the TIMSS 2011 
International Results in Mathematics (Mullis, Martin, Foy, & Arora, 2012), the 
average percent correct across the mathematics items overall was 50 percent, 
with 47 percent correct, on average, for the number items, 49 percent for the 
geometric shapes and measures items, and 58 percent for the data display 
items. On other hand, the degree of reading demands in the mathematics items 
by content domains has some relationship with the emphasis on the topics 
in these three content domains in the curricula across countries. According 
to data published in the TIMSS 2011 International Results in Mathematics, 
across countries, National Research Coordinators reported that of the eight 
number topics, on average, six (75%) were included in the curriculum; of the 
seven geometry topics, five (71%) were included, and of the three data display 
topics, two (67%) were included. According to their teachers, the percentage of 
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students that had been taught the TIMSS topics, averaged across each content 
domain, was lower for geometric shapes and measures (65%) than for both 
number and for data display (76%). The NRC and teacher data indicate less 
curricular emphasis on the TIMSS topics in geometric shapes and measures 
than in number, whereas the TIMSS items in geometric shapes and measures 
are more likely than the number items to have medium rather than low reading 
demands. The situation with data display is more difficult to interpret because 
it receives a small emphasis in the assessment (15%) and only has three topics. 
Although the three data display topics seem to be present in the curricula and 
classrooms of the TIMSS countries, this probably represents only a small part 
of students’ instruction in mathematics and the TIMSS items in this content 
domain are likely to have relatively high reading demands. 

Looking across the science content domains, the life science items — 
covering topics about the characteristics, processes, and cycles of living things 
and comprising nearly half of the assessment (45%) — were relatively well 
distributed according to reading demand, although with a tendency toward 
lower or medium rather than high levels of reading difficulty. The opposite was 
found for the physical science items (35% of the assessment) which often involve 
physical phenomena that can be presented via models or diagrams. Thus, items 
in the physical science content domain were more likely to have medium and 
high reading demands. Finally, the one-fifth of the assessment devoted to earth 
science was well-balanced with most items categorized as having medium 
reading demands, but some with low and some with high reading demands. 

For science at the fourth grade, as reported in TIMSS 201 1 International 
Results in Science (Martin, Mullis, Foy, & Stanco, 2012) the average percent 
correct across countries was 48 percent overall, and very similar across content 
domains — life science (48%), earth science (46%), and physical science (49%). 
However, the content areas more likely to have TIMSS items with high reading 
demands were the content areas emphasized least in the curricula across 
countries. The life science items were most likely to have low reading demands, 
earth science items medium reading demands, and physical science items high 
reading demands. This corresponds with the curricular emphasis on the science 
content areas, with life science being emphasized more than earth science, and 
earth science, in turn, emphasized more the physical science. As presented in the 
TIMSS 201 1 International Results in Science, across countries, National Research 
Coordinators reported, on average, that of the six life science topics, five (84%) 
were included in the curriculum for all students; of the six earth science topics, 
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four (67%) were included; and of the eight physical science topics, five (63%) 
were included. Consistent with the country reports, according to their teachers, 
on average, the percentage of students taught the TIMSS topics was highest for 
life science (75%), next highest for earth science (63%), and lowest for physical 
science (57%). 

Exhibit 2.9 presents the distributions by cognitive domain of the TIMSS 
2011 mathematics and science fourth grade items categorized as having low, 
medium, and high reading demands. In general, the patterns are similar for 
mathematics and science. The two-fifths of the items measuring the knowing 
domain (e.g., recall, recognize, compute, classify/order) were more likely to 
have low reading demands (61% in mathematics and 56% in science); the two- 
fifths of the items measuring the applying domain (e.g., represent, model, and 
solve standard problems) were more likely to have medium reading demands; 
and the one-fifth of the items measuring reasoning (e.g., analyze, synthesize, 
justify, and solve problems in unfamiliar or complex contexts) were most likely 
to have high reading demands (59% in mathematics and 76% in science). Items 
measuring students’ ability to apply their knowledge of content and procedures 
and, especially those requiring reasoning, need to include some information 
in the form of words or visual displays as to the problems situation. The items 
measuring reasoning often were based on scenarios or situations reflecting 
school or daily experiences. 

Exhibit 2.9: Percentage of TIMSS Fourth Grade Mathematics and Science Items by Cognitive Domains 
and Low, Medium, and High Reading Demands 



For the cognitive domains in mathematics and science, the items were 
more difficult across the cognitive areas from knowing, to applying, and then 
reasoning. For mathematics, the average percent correct across countries was 
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55 percent for the items classified in the knowing domain, 50 percent for items 
classified in the applying domain, and substantially lower for items in the 
reasoning domain — 40 percent (Mullis, Martin, Foy, & Arora, 2012). In science, 
the average percent correct across countries was 53 percent for knowing, 
46 percent for applying, and 41 percent for reasoning (Martin, Mullis, Foy, & 
Stanco, 2012). Certainly, it can be considered that both the complexity of the 
cognitive tasks and the increase in reading demands both contribute to the 
substantial difficulty of the reasoning items. 

Generalizability Across Countries of the Item 
Categorizations According Low, Medium, and 
High Reading Demands 

Because the study of the impact of item reading demands on TIMSS fourth 
grade mathematics and science achievement was conducted using the English 
language version of the items, the question arises about the generalizability 
of the results to languages other than English. Clearly, the four indicators of 
reading demands would not be expected to have identical values in all languages. 
For example, it is well known that after translation the number and length of 
words in the TIMSS items varies across languages. However, it is likely that the 
items with the highest reading demands in English also would have the highest 
reading demands in other languages. Thus, National Research Coordinators 
of countries that conducted TIMSS 2011 in languages other than English 
participated in a Reading Demands Matching Analysis (RDMA) which involved 
categorizing the items in their languages into three categories of reading 
demand (highest, medium, and lowest). NRCs were instructed to categorize 
the items in their languages using a holistic rating process based on the same 
four indicators of reading demand used in the study (number of words, number 
of different symbols, number of different specialized vocabulary terms, and 
number/density of visual displays). Seventeen countries and one benchmarking 
participant (representing 16 different languages in total) submitted their RDMA 
categorizations to the TIMSS & PIRLS International Study Center. On average, 
NRCs reported involving four different raters in the RDMA process. The rating 
teams included members of their TIMSS or PIRLS teams and/or content area 
experts, including researchers, curriculum experts, and teachers. A number of 
countries reported that they valued participating in the process and appreciated 
viewing the TIMSS items from a different perspective. 
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The categorizations submitted by the NRCs were compared to the 
categorizations developed by the TIMSS & PIRLS International Study Center, 
and there was a very a high degree of agreement for both the mathematics and 
science items. On average, there was 71 percent exact agreement and 98 percent 
adjacent agreement across countries. Although there were some items (15%) 
where the level of exact agreement was below 50 percent, these were often items 
with complicated graphics. 

The Impact of Reading Ability on TIMSS Achievement for 
Items with Low, Medium, and High Reading Demand 

Keeping in mind that the level of reading demands in the TIMSS fourth grade 
mathematics and science items interacts with many other factors, including 
the difficulty and curriculum coverage of the topics in the content domains as 
well as the variation in difficulty across the cognitive domains, it is still very 
interesting to look at performance on the TIMSS mathematics and science 
items for students of different reading ability and how this relates to the level of 
reading demands in the items from low to medium to high. 

The relationship between level of reading ability and TIMSS mathematics 
and science achievement by level of reading demand in the items was examined 
by computing the average percent correct 1 for items in each of the three 
categorizations of reading demands (low, medium, and high) for students with 
three levels of reading ability. More specifically, separately for mathematics 
and for science, the TIMSS & PIRLS International Study Center computed 
the average percent correct for each group of the fourth grade items classified 
according to low, medium, and high reading demands for each of three levels 
of students’ reading ability (determined by the lower, middle, and upper terciles 
of reading achievement on PIRLS 2011 in each country). 

It was expected that the best readers would be unaffected by the reading 
demands of the items, and therefore would perform similarly on the low, 
medium, and high demand items, whereas the poorest readers would perform 
relatively better on low demand items, and less well on high demand items. 
Some support for these hypotheses was found in the overall and country- 
by-country results, particularly in mathematics. However, the results varied 
considerably from country to country and even between mathematics and 
science within countries. 


1 The analyses also were conducted based on achievement scales created for each of the three groups of items using the 
same scaling approach as used for the TIMSS 2011 content and cognitive domains (i.e., a multi-dimension estimation 
of performance on high, medium, and low reading demand items using the item parameters from the TIMSS overall 
concurrent calibration). However, the average percent correct approach used in this paper seemed to provide more easily 
interpretable results. 
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Exhibit 2.10 presents line graphs of the international averages across the 
country-by- country results for TIMSS fourth grade the mathematics items, and 
Exhibit 2.1 1 presents the same information for the science items. As would be 
expected, Exhibit 2.10 for mathematics shows that across the three levels of 
reading demands (low, medium, and high), the students in the upper reading 
tercile (top one-third of readers averaged across countries, indicated by circles) 
had higher average mathematics achievement than those in the middle reading 
tercile (indicated by triangles). In turn, the students in the middle reading tercile 
had higher average mathematics achievement at each level of item reading 
demand than did those in the lower reading tercile (indicated by squares). 

Exhibit 2.10: Mathematics Achievement Averaged Across Countries — Fourth Grade 

Average Percent Correct for Students at Three Levels ofPIRLS Reading Ability on 
Mathematics Items Grouped by Three Levels of Reading Demands 
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As hypothesized, the average mathematics achievement of the best 
readers did not vary much by level of reading demands (66% correct on the 
low demand items, 63% on the medium items, and 66% on the high items). In 
comparison, the average mathematics achievement of the least proficient readers 
was 39 percent correct on the items with low reading demands, but lower on 
the items of medium (34%) and high reading demands (33%). The difference 
in average achievement between poor and good readers on the low reading 
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demand items was 27 percentage points. However, this difference increased 
to 32 percentage points on the high reading demand items. While the poorest 
readers consistently achieved at a lower level in mathematics than the best 
readers, they were additionally disadvantaged on the mathematics items that 
required more reading. 

Exhibit 2.11 for science shows results for the poorest readers that are nearly 
identical to those shown for mathematics, with averages of 39 percent correct 
on the low demand items and 33 percent correct on both the medium and high 
demand items. Also, the achievement gaps between the three terciles of readers 
on the science items with low reading demands are nearly identical to those for 

Exhibit 2.1 1 : Science Achievement Averaged Across Countries — Fourth Grade 

Average Percent Correct for Students at Three Levels ofPIRLS Reading Ability on 
Science Items Grouped by Three Levels of Reading Demands 
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mathematics. However, for the science items, similar to the poorest tercile of 
readers, the upper and middle terciles of readers also had lower achievement on 
the medium and high reading demand items. Interestingly for science, there was 
little difference in the results between the medium and high reading demand 
items, which may reflect a small difference in reading demands between the 
medium and high demand items. Because all three terciles of readers were 
similarly disadvantaged by more reading demands, the gaps in achievement 
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between the upper tercile of readers compared to the lower tercile of readers 
was similar at all three levels of reading demand (28% on the low demand and 
medium demand items, and 29% on the high demand items). 

Exhibit 2.12 presents the TIMSS fourth grade mathematics achievement 
results by level of reading demand and tercile of PIRLS reading achievement 
for each of the 34 countries and 3 benchmarking entities that assessed the 
same fourth grade students with TIMSS and PIRLS in 2011. Looking across 
countries, in most instances there is a significant difference in average percent 
correct between the upper and lower reading achievement terciles on the low 
reading demand items (left side of graphs) and the high reading demand items 
(right side of graphs) with the difference being larger on the high reading 
demand items. That is, for most countries, better readers have a significantly 
greater advantage over poorer readers on mathematics items with high reading 
demands. 

The significant difference in the achievement gap between low and high 
reading demand items seemed to arise from the expected pattern — the best 
readers having similar mathematics achievement across all items regardless of 
level of reading demands, but poor readers performing less well on items with 
more reading demands — most noticeably in Australia, Austria, Chinese Taipei, 
Croatia, Hungary, Lithuania, Portugal, the Russian Lederation, Saudi Arabia, 
and the Slovak Republic. 
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Exhibit 2.1 2: Mathematics Achievement, Country by Country — Fourth Grade 

Average Percent Correct for Students at Three Levels ofPIRLS Reading Ability on 
Mathematics Items Grouped by Three Levels of Reading Demands 
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Exhibit 2.12: Mathematics Achievement, Country by Country — Fourth Grade (Continued) 

Average Percent Correct for Students at Three Levels ofPIRLS Reading Ability on 
Mathematics Items Grouped by Three Levels of Reading Demands 
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Exhibit 2.1 2: Mathematics Achievement, Country by Country — Fourth Grade (Continued) 

Average Percent Correct for Students at Three Levels ofPIRLS Reading Ability on 
Mathematics Items Grouped by Three Levels of Reading Demands 
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Exhibit 2.1 2: Mathematics Achievement, Country by Country — Fourth Grade (Continued) 

Average Percent Correct for Students at Three Levels ofPIRLS Reading Ability on 
Mathematics Items Grouped by Three Levels of Reading Demands 
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Exhibit 2 . 1 2 : Mathematics Achievement, Country by Country — Fourth Grade (Continued) TIMSS & PHtLS 

Average Percent Correct for Students at Three Levels ofPIRLS Reading Ability on 
Mathematics Items Grouped by Three Levels of Reading Demands 
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Exhibit 2.1 2: Mathematics Achievement, Country by Country — Fourth Grade (Continued) 

Average Percent Correct for Students at Three Levels ofPIRLS Reading Ability on 
Mathematics Items Grouped by Three Levels of Reading Demands 
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Exhibit 2.1 2: Mathematics Achievement, Country by Country — Fourth Grade (Continued) 

Average Percent Correct for Students at Three Levels ofPIRLS Reading Ability on 
Mathematics Items Grouped by Three Levels of Reading Demands 
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Exhibit 2.13 presents the TIMSS fourth grade science achievement results 
by level of reading demand and tercile of PIRLS reading achievement for each 
of the 34 countries and 3 benchmarking entities that assessed the same fourth 
grade students with TIMSS and PIRLS in 2011. Again, the results may reflect 
the restricted range in the level of reading demands in the fourth grade science 
items, or that at the fourth grade science content and reading are very closely 
linked. Consistent with results averaged across countries (Exhibit 2.11), there 
were fewer significant differences than in mathematics between the achievement 
gap for low reading demand items and high reading demand items that show 
an advantage for better readers over poorer readers on the high reading 
demand items. Elowever, such significant differences were found in more 
than half of the countries, including Croatia, the Czech Republic, Hungary, 
Lithuania, Poland, Romania, the Russian Federation, the Slovak Republic, and 
Slovenia where the difference was 5 or more percentage points. Interestingly, 
compared to mathematics, there were more countries, such as Azerbaijan, 
Georgia, Iran, Morocco, Norway, Romania, and Saudi Arabia, where increased 
reading demands in the science items was systematically associated with lower 
achievement for the three levels of readers. In addition, there were a number 
of countries where performance dropped by about the same amount on the 
medium and high reading demand items for all three levels of readers. 
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Exhibit 2.1 3: Science Achievement, Country by Country — Fourth Grade 

Average Percent Correct for Students at Three Levels ofPIRLS Reading Ability on 
Science Items Grouped by Three Levels of Reading Demands 
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Exhibit 2.1 3: Science Achievement, Country by Country — Fourth Grade (Continued) 

Average Percent Correct for Students at Three Levels ofPIRLS Reading Ability on 
Science Items Grouped by Three Levels of Reading Demands 
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Exhibit 2.1 3: Science Achievement, Country by Country — Fourth Grade (Continued) TIMSS fr PIJRLS 

Average Percent Correct for Students at Three Levels ofPIRLS Reading Ability on 
Science Items Grouped by Three Levels of Reading Demands 
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*The inter-tercile difference for High Reading Demands is significantly different from the inter-tercile difference for Low Reading Demands. 
Because of rounding, some results may appear inconsistent. 
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Exhibit 2.1 3: Science Achievement, Country by Country — Fourth Grade (Continued) 

Average Percent Correct for Students at Three Levels ofPIRLS Reading Ability on 
Science Items Grouped by Three Levels of Reading Demands 
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*The inter-tercile difference for High Reading Demands is significantly different from the inter-tercile difference for Low Reading Demands. 
Because of rounding, some results may appear inconsistent. 
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Exhibit 2.1 3: Science Achievement, Country by Country — Fourth Grade (Continued) TIMSS fr PIJRLS 

Average Percent Correct for Students at Three Levels ofPIRLS Reading Ability on 
Science Items Grouped by Three Levels of Reading Demands 
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Because of rounding, some results may appear inconsistent. 
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Exhibit 2.1 3: Science Achievement, Country by Country — Fourth Grade (Continued) 

Average Percent Correct for Students at Three Levels ofPIRLS Reading Ability on 
Science Items Grouped by Three Levels of Reading Demands 
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•The inter-tercile difference for High Reading Demands is significantly different from the inter-tercile difference for Low Reading Demands. 
Because of rounding, some results may appear inconsistent. 
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Exhibit 2.1 3: Science Achievement, Country by Country — Fourth Grade (Continued) 

Average Percent Correct for Students at Three Levels ofPIRLS Reading Ability on 
Science Items Grouped by Three Levels of Reading Demands 
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Because of rounding, some results may appear inconsistent. 
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Considering the Results 

This study hypothesized that the best readers would be unaffected by the 
reading demands of the items and therefore would perform similarly on the low, 
medium, and high demand items, whereas the poorest readers would perform 
relatively better on low demand items, and less well on high demand items. 
Some support for these hypotheses was found in the overall and country-by- 
country results, particularly in mathematics. 

On average across countries, the mathematics achievement difference 
between poor and good readers was larger on the high reading demand items 
than on the low reading demand items. The average mathematics achievement 
of the best readers did not vary much by level of reading demands, whereas the 
average mathematics achievement of the least proficient readers was higher on 
the items with low reading demands than on the items with medium and high 
reading demands. While the poorest readers consistently achieved at a lower 
level in mathematics than the best readers, they were additionally disadvantaged 
on the mathematics items that required more reading. 

Also, looking at the results for mathematics country by country, in nearly 
all instances the difference in average achievement between poor and good 
readers was larger on the high reading demand items than on the low reading 
demand items. That is, for most countries, better readers have a significantly 
greater advantage over poorer readers on mathematics items with high reading 
demands. 

On average across countries in science, all three terciles of readers were 
similarly disadvantaged by more reading demands, so the gaps in achievement 
between the upper tercile of readers compared to the lower tercile of readers was 
similar at all three levels of reading demands. In addition, although achievement 
was higher on the items with low reading demands, there was little difference in 
the results between the medium and high reading demand items. 

The country-by-country results in science showed that the difference in 
average achievement between poor and good readers was larger on the high 
reading demand items than on the low reading demand items in more than 
half of the countries. However, the difference also was approximately similar in 
a number of countries. 

In summary, much was learned from this research regarding the challenges 
of educational measurement. Perhaps as a by-product, though still important, 
was developing procedures to look at the characteristics of the TIMSS fourth 
grade items through the lens of reading difficulty. In addition to the already 
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lengthy checklists used to review items for various aspects of content validity 
and clarity, it is important to scrutinize the TIMSS items from the perspective 
of “mathematics reading” or “science reading” (e.g., the number of words, the 
number of different symbols, the load of the technical vocabulary, and the roles 
that are being played by visual displays). 

More important, however, was gaining a deeper understanding of the 
extreme complexity of the educational endeavor and all of its interconnected 
parts. Students all over the world are learning mathematics and science, but they 
are learning these subjects in different ways. In particular, as this research has 
highlighted, curricular and instructional differences experienced by students 
can impact item difficulty. As we know, countries’ mathematics and science 
curricula vary considerably, and the different amounts of emphasis placed on 
the topics covered has a powerful influence on student learning as well as on 
student achievement on TIMSS. Beyond that, instructional differences among 
countries not only affect achievement in the content domains, but also in the 
cognitive domains. There is an interrelationship between cognitive domain and 
reading demands, insofar as assessing in-depth content understanding and 
increased cognitive complexity generally involves greater reading demands; 
thus, TIMSS mathematics and science items become more difficult for a variety 
of reasons. 

Especially relevant to educational research and policy broadly, though, 
is that reading is fundamental to further learning. It makes good sense that 
students who are better readers are therefore better positioned to learn more in 
mathematics and science as well as in their other subjects. Although the results 
of this study varied considerably from country to country, and even between 
mathematics and science within countries, the study showed reading ability 
to be associated with mathematics and science achievement to an extent that 
provides support for the idea that greater reading demands can make the fourth 
grade TIMSS items more challenging for weaker readers. Extrapolating this idea 
into the broader educational arena raises the question of how much reading 
intervention might influence learning across the curriculum. 
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Chapter 3 


Effective Schools in Reading, 
Mathematics, and Science at the 
Fourth Grade 

Michael 0. Martin, Pierre Foy, Ina V.S. Mullis, and Laura M. O'Dwyer 1 

Boston College 

Introduction 

Results from several cycles of TIMSS and PIRLS as well as 
from considerable research conducted around the world have 
demonstrated the consistency of a number of fundamental 
school factors being associated with higher achievement across 
the school subjects of reading, mathematics, and science. Data 
gathered from the 34 countries and 3 benchmarking entities 
that conducted the TIMSS and PIRLS 2011 assessments with 
the same fourth grade students present a unique opportunity to 
study relative school effectiveness across countries in reading, 
mathematics, and science. This chapter examines how these 
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important school factors operate across countries at the fourth grade and vary 
in an international context. 

The TIMSS 2011 fourth grade assessments in mathematics and science and 
the PIRLS 2011 assessment in reading comprehension provide comprehensive, 
robust achievement measures in these three core learning areas. The assessments 
were based on comprehensive frameworks developed collaboratively with the 
participating countries, and included large numbers of items to assess these 
frameworks (175 items in mathematics, 168 items in science, and 130 items 
in reading). For each country, the data are based on nationally representative 
samples of students. Altogether, nearly 200,000 students took part in TIMSS and 
PIRTS 2011, with each student assessed in all three of the core school subjects. 

In addition to comprehensive assessments of achievement, TIMSS 
and PIRTS 2011 each included an array of background questionnaires to 
collect information about the contexts for teaching and learning in each of 
the participating countries. The context questionnaires administered to 
school principals, teachers, and students were designed to collect a range of 
information about school environments, school resources, teacher preparation, 
and classroom instruction. In addition, a questionnaire for parents collected 
information about students’ home environment. The TIMSS and PIRLS 2011 
participants jointly developed the questionnaires, and the response data were 
used to create new TIMSS and PIRLS context questionnaire scales using IRT 
methods. These scales measure important school factors, such as academic 
climate, resource adequacy, school safety, curricular emphasis, and instructional 
engagement, as well as aspects of the home environment. The context 
questionnaire scales were developed in parallel across reading, mathematics, and 
science, and provide a solid foundation for studying the relative effectiveness of 
school and classroom contexts for teaching and learning. 

This chapter is intended to illustrate the power of the TIMSS and 
PIRLS 2011 data for studying school effectiveness by exploring relationships 
among school environment and instruction, student home background, and 
student achievement in reading, mathematics, and science in the participating 
countries. As such, the study is not intended to be a definitive analysis of the 
factors associated with effective schools in each country. Rather, this chapter 
presents an analytic framework that could provide an overview of how these 
relationships vary across countries and be used as a starting point for more 
detailed analyses within countries. 
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School Effectiveness Analyses 

School effectiveness analyses seek to improve educational practice by studying 
what makes for a successful school beyond having a student body where most 
of the students are from advantaged socioeconomic backgrounds. Using this 
approach, an effective school is one that has an effect on student achievement 
over and above home influences. From an analytic perspective, school 
effectiveness studies make use of multilevel modeling in order to analyze the 
relationship between school factors and achievement after controlling for the 
influences of students’ home backgrounds. Because TIMSS and PIRLS 2011 
included a home questionnaire completed by students’ parents and primary 
caregivers, the data provide considerable information about students’ home 
environments. This study uses these data to examine the effects of home 
environment on students’ achievement and then to control for those effects 
in looking at the school factors. Examining students’ schooling in the light of 
home factors can help to better understand the interaction between home and 
school effects. 

According to Sammons (2007), researchers have been examining different 
aspects of school effectiveness in order to improve educational outcomes for 
students for more than 40 years; further, while definitions of school effectiveness 
vary, most researchers agree that, when comparing schools with similar student 
populations, an effective school is one that “adds extra value” to student 
achievement. That is, the characteristics students have when entering school 
are strongly associated with achievement and should be explicitly controlled 
in the analysis model in order to better isolate the effects of a school. An 
effective school has the capacity to improve students’ achievement despite the 
characteristics of the student body. 

Teddlie and Reynolds (2000) have traced the origins of school effective 
research back to the mid-1960s in the United States when most educational 
research involved investigating the relationship between inputs (human and 
physical resources) and outputs (student achievement). A school was defined 
by its material resources, and differences in student achievement were attributed 
to unequal opportunities in terms of school environments (Scheerens & Bosker, 
1997). However, because school differences primarily were attributed to student 
background characteristics rather than educational practices, educational 
researchers were criticized for not measuring the educational processes within 
schools (Teddlie & Reynolds, 2000). In addition, the lack of sophisticated 
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methodology prevented researchers from making fair comparisons between 
schools (Creemers, Kyriakides, & Sammons, 2010). 

In the 1980s, more contextual factors (e.g., psychological scales) and 
more sensitive outcome measures were used in school effectiveness research 
(Townsend, 2007). Also, advances in computing technology had made computer 
programs for multilevel modeling more widely available (Teddlie & Reynolds, 
2000). The most notable improvement in school effectiveness research, 
according to Rumberger & Palardy (2004), was using multilevel modeling to 
estimate the effects of factors on student outcomes more accurately, by looking 
at the effects at different levels in the education system (i.e., the student level, 
classroom level, and school level). 

School effectiveness research distinguishes itself from other strands of 
educational effectiveness research, such as economically oriented studies 
and instructional effectiveness studies, by focusing on the importance of 
differences between schools (Scheerens & Bosker, 1997). Research investigating 
school differences focuses on the aspects of schools that have an influence on 
achievement so that the results can be used to suggest improvements and shape 
reform policies. As such, school effectiveness research uses the school as the 
major unit of change in educational reform (Teddlie, 2010). 

A Strong Conceptual Model 

Like all studies of cross-sectional survey data, the statistical modeling conducted 
for this study is crucially dependent on the naturally-occurring variation and 
covariation in the data. The assumption is that expected relationships between 
variables will be reflected in observable patterns in the data. However, education 
systems are the result of management and development over many years, and 
often the variables of interest have been manipulated to achieve policy goals, 
so that expected relationships may not be apparent in the data. For example, 
the school system may be organized so that there is little difference between 
schools in the achievement of their students, making it difficult to relate school 
factors to student achievement. Similarly, there may be particular factors, such 
as instructional time, that are the same for all the schools in a country and 
consequently cannot play a role in a statistical model for that country. In this 
situation, it is important to have a strong conceptual model based on a clear 
vision of the essential characteristics of effective schools in order to guide 
the analysis. 
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This study of school effectiveness is deeply rooted in considerable 
work studying the factors that influence school quality, as documented in 
the TIMSS 2011 and PIRLS 2011 Contextual Frameworks (Mullis, Martin, 
Ruddock, O’Sullivan, & Preuschoff, 2009; Mullis, Martin, Kennedy, Trong, & 
Sainsbury, 2009). Building on that body of research, the TIMSS 2011 Results 
in Mathematics (Mullis, Martin, Foy, & Arora, 2012), the TIMSS 2011 Results 
in Science (Martin, Mullis, Foy, & Stanco, 2012), and the PIRLS 2011 Results 
in Reading (Mullis, Martin, Foy, & Drucker, 2012) showed that students with 
the highest achievement typically attended schools that emphasize academic 
success, as indicated by rigorous curricular goals, effective teachers, students 
that desire to do well, and parental support. Students that attended schools with 
disorderly environments and reported more frequent bullying had much lower 
achievement than their counterparts in safe and orderly schools. Exhibit 3.1 
presents this study’s conceptual framework of school effectiveness, informed 
by the latest results from TIMSS and PIRLS 201 1 as well as the existing body of 
school effectiveness research. 

Exhibit 3.1 : Model of Effective Schools 


Strongly supported by the research, this study 
maintains a firm conviction that effective schools: 


Are Safe and Orderly 
Support Academic Success 
Have adequate facilities and equipment 
Are staffed with well-prepared teachers 
Have well-resourced classrooms 
Provide effective instruction 


First, TIMSS and PIRLS 2011 results showed that students who attended 
schools with disorderly environments and who reported more frequent bullying 
had much lower achievement than their counterparts in safe and orderly 
schools. It makes sense that for a school to be effective, it needs a safe and 
orderly environment, and that schools with considerable disciplinary problems 
are not conducive to higher student achievement. When students and teachers 
are fearful and worried about their safety, it is difficult to maintain a strong 
focus on academics. 

Second, students with the highest achievement in TIMSS and PIRLS 2011 
typically attended schools that emphasized academic success. In order to achieve 
excellence, it is not enough to simply “keep order”; the school administration 
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and teachers as well as the students and their parents must press for academic 
success. A school with a positive atmosphere supportive of high achievement 
and a rigorous academic program can even overcome socioeconomic 
disadvantages (McGuigan & Hoy, 2006). From this perspective, schools need 
to communicate their academic emphasis through clear and rigorous academic 
goals, and school administrators and teachers need to support these goals and 
believe that students can attain them. The effect of aiming for students’ high 
achievement is greatest when there is a collective influence, including parents 
and the students themselves. 

Third, TIMSS and PIRLS and many other studies have shown that 
resources are crucial for effective schooling, perhaps even more so in developing 
countries than in economically developed countries. The 2011 data showed 
that students in schools not affected by resource shortages had higher average 
achievement than their counterparts in less well-resourced schools. Successful 
schools are likely to have better working conditions and facilities, such that 
the physical environment is structurally sturdy, big enough, well-maintained, 
and comfortable (e.g., temperature and lighting). Additionally, effective schools 
have more instructional materials, such as books, computers, technologically 
innovative instructional aids, and equipment and supplies (everything from 
basic paper and pencils to science laboratory equipment). 

Fourth, although the school environment and facilities can provide 
important support for teaching and learning, teacher quality is essential 
because most instruction is provided directly to students by classroom teachers. 
There is growing evidence that teacher preparation is a powerful predictor of 
students’ achievement, perhaps even overcoming socioeconomic and language 
background factors (Darling- Hammond, 2000). To engage students in learning, 
teachers need to be well-prepared such that they have a solid mastery of the 
content in the subject to be taught and a repertoire of effective pedagogical 
approaches. In TIMSS 2011, higher achievement was related to teachers’ having 
more teaching experience, being confident in their teaching, and being satisfied 
with their careers. The majority of fourth grade students had teachers with 
bachelor’s degrees, and most had teachers that reported having at least ten years 
of teaching experience, being very well prepared to teach their subject matter, 
and feeling very confident in their teaching. 

Fifth, teachers need well-resourced classrooms. There are many 
resources that can facilitate classroom teaching, such as textbooks, computers, 
instructional software, and equipment for various projects. There also are 
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some materials associated with instruction in specific subjects. For example, 
because having students read books and a variety of different types of materials 
is fundamental to developing their reading comprehension skills and strategies, 
a number of educational systems have invested in classroom libraries so 
that children can have ready access to books and magazines as part of their 
reading lessons and activities. Similarly, access to calculators and a variety 
of manipulative materials can be important to mathematics instruction, and 
science equipment, models, and materials can be central to science instruction. 

Sixth, teachers need to provide effective, engaging instruction. Teachers 
who are well-prepared and well-resourced most likely would be in the best 
position to provide effective, engaging instruction. According to work supported 
by the US Center for Education Statistics (McLaughlin et al., 2005), student 
engagement focuses on the importance of the activity that brings the student 
and the subject matter content together. Engagement refers to the cognitive 
interaction between the students and instructional content, and this interaction 
can be stimulated by any instructional approach. What matters is students’ 
in-the-moment cognitive interaction with the instructional content such that 
learning takes place. Of course, if students are being involved in the instruction 
in some way, even by attentive listening, then there is a much higher likelihood of 
engagement and learning. In TIMSS and PIRLS 2011, internationally, the fourth 
grade students who were “engaged” in their mathematics, science, and reading 
lessons had the highest achievement, followed by those “somewhat engaged,” 
and then those few students “not engaged” with much lower achievement. 

Measures of School Effectiveness 

The large amount of background data collected in TIMSS and PIRLS 2011 was 
reviewed in order to select the school effectiveness measures to include in the 
present study. As a fundamental selection criterion, the measures needed to 
address school characteristics included in the conceptual model. In addition, 
because the idea was to study relative effects across reading, mathematics, and 
science, the measures needed to be available in parallel across all three subjects. 
Also, as much as possible, measures needed to be consistently related to higher 
achievement at the fourth grade across the three subject areas and across the 
participating countries. Finally, as matter of reliability and efficiency, it was 
decided primarily to use or modify scales that were included in the TIMSS and 
PIRLS 2011 international reports. 
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In the end, 1 1 context questionnaire scales were selected for inclusion in 
the analysis and combined into five robust school effectiveness measures: three 
measures of effective school environment, and two measures of effective school 
instruction. In general, each of the five measures of school effectiveness was 
based on a school average of several context questionnaire scales, with each 
context questionnaire scale typically including about six questions/statements. 
The components of the five school effectiveness measures are described in the 
following sections, with further detail available in Technical Appendix B: School 
Effectiveness Models and Analyses. 

School Environment 

Following this study’s conceptual model, the school effectiveness analyses 
include three measures of school environment: 

♦ Schools are safe and orderly; 

♦ Schools support academic success; and 

♦ Schools have a physical environment and resources that are 
adequate for learning. 

Schools Are Safe and Orderly 

The measure of a school being safe and orderly was the school average of three 
different context questionnaire scales measuring school safety: one based on 
principals’ reports of discipline problems in the school, one based on teachers’ 
reports of school safety, and one based on students’ reports of being bullied. 

School Discipline and Safety Scale— Principals’ reports of “hardly any 
problems,” “minor problems,” or “moderate problems” concerning 10 potential 
school problems: 

♦ Students arriving late at school; 

♦ Student absenteeism (i.e., unjustified absences); 

♦ Classroom disturbances; 

♦ Cheating; 

♦ Profanity; 

♦ Vandalism; 

♦ Theft; 

♦ Intimidation or verbal abuse among students (including texting, 
emailing, etc.); 

♦ Physical fights among students; and 

♦ Intimidation or verbal abuse of teachers or staff (including texting, 
emailing, etc.). 
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Safe and Orderly School Scale — Teachers’ degree of agreement with the 
following five statements: 

♦ This school is located in a safe neighborhood; 

♦ I feel safe at this school; 

♦ This school’s security policies and practices are sufficient; 

♦ The students behave in an orderly manner; and 

♦ The students are respectful of the teacher. 

Students Bullied at School Scale — Students’ reports about how often they 
experienced the following six bullying behaviors: 

♦ I was made fun of or called names; 

♦ I was left out of games or activities by other students; 

♦ Someone spread lies about me; 

♦ Something was stolen from me; 

♦ I was hit or hurt by other students (e.g., shoving, hitting, kicking); 
and 

♦ I was made to do things I didn’t want to do by other students. 

Schools Support Academic Success 

The measure of a school’s degree of support for academic success was the school 
average of two context questionnaire scales. In this case, teachers and principals 
provided responses to the same emphasis on academic success scale. 

Teachers’ Reports, School Emphasis on Academic Success— Teachers’ 
responses characterizing five aspects of their school as Very High, High, or 
Medium: 

Principals’ Reports, School Emphasis on Academic Success— Principals’ 
responses characterizing five aspects of their school as Very High, High, or 
Medium: 

♦ Teachers’ understanding of the curricular goals; 

♦ Teachers’ degree of success in implementing the school’s 
curriculum; 

♦ Teachers’ expectation for student achievement; 

♦ Parental support for student achievement; and 

♦ Students’ desire to do well in school. 
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Adequate Environment and Resources 

This effective schools measure was the school average of two context 
questionnaire scales: one based on teachers’ views of their working conditions, 
and the other based on principals’ perceptions of the degree to which the school 
facilities and resource availability were affecting the quality of instruction. 

Teacher Working Conditions Scale — Teachers’ reports that they had 
“Hardly Any Problems,” “Minor Problems,” or “Moderate Problems” concerning 
five potential problem areas: 

♦ The school building needs significant repair; 

♦ Classrooms are overcrowded; 

♦ Teachers have too many teaching hours; 

♦ Teachers do not have adequate workspace (e.g., for preparation, 
collaboration, or meeting with students); and 

♦ Teachers do not have adequate instruction materials and supplies. 

School Resource Shortage Scale — Principals’ responses about whether 
instruction was “Not Affected,” “Somewhat Affected,” or “Affected A Lot” by 
resource shortages in four areas: A. general school resources (six questions); 
B. mathematics resources (six questions); C. science resources (six questions); 
and D. reading resources (five questions). Although this entire set of questions 
was presented together to principals and analyzed as a single scale for this 
analysis, three separate scales were created for the three international reports 
containing the mathematics, science, and reading results (i.e., the general and 
mathematics questions, the general and science questions, and the general and 
reading questions). 

Section A — General School Resources 

♦ Instructional materials (e.g., textbooks); 

♦ Supplies (e.g., paper, pencils); 

♦ School buildings and grounds; 

♦ Heating/cooling and lighting systems; 

♦ Instructional space (e.g., classrooms); and 

♦ Technologically competent staff. 

Sections B, C, and D — Subject specific resources (Mathematics, Science, 
and Reading, respectively). Sections B, asking about mathematics, and C, 
asking about science, contained the same six questions. Section D, asking about 
reading, contained five of the six questions, the exception being “calculators for 
instruction”: 
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♦ Teachers with a specialization in the subject; 

♦ Computers for instruction; 

♦ Computer software for instruction; 

♦ Library materials for instruction; 

♦ Audio-visual resources for instruction; and 

♦ Calculators. 

School Instruction 

Selecting school effectiveness measures of classroom instruction was more 
challenging, because the TIMSS and PIRLS 2011 background data and context 
questionnaire scales about teacher quality and instructional engagement are less 
well developed than the data about school climate and resources. However, two 
indicators of instructional effectiveness are included in the school effectiveness 
models. The school effectiveness measures of school instructional quality are 
as follows: 

♦ Early curricular emphasis on higher order reading processes; and 

♦ Students engaged in reading, mathematics, and science lessons. 

Early Curricular Emphasis on Reading Skills 

This measure is one of the context questionnaire scales developed for 
PIRLS 2011. 

Emphasis in Early Grades on Reading Skills and Strategies— Principals’ 
responses about the earliest grade at which each of 11 reading skills and 
strategies were emphasized. 

♦ Reading isolated sentences; 

♦ Reading connected text; 

♦ Locating information within the text; 

♦ Identifying the main idea of a text; 

♦ Explaining or supporting understanding of a text; 

♦ Comparing a text with personal experience; 

♦ Comparing different texts; 

♦ Making predictions about what will happen next in a text; 

♦ Making generalizations and drawing inferences based on a text; 

♦ Describing the style or structure of a text; and 

♦ Determining the authors perspective or intention. 
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Student Engaged in Reading, Mathematics, and Science Lessons 
The measure of student engagement in instruction was the school average across 
three context questionnaire scales measuring students’ engagement in their 
lessons: one for reading, one for mathematics, and one for science. Students 
reported separately about their mathematics and science lessons in terms of 
their degree of agreement with the five statements listed below: 

Students Engaged in Lessons — Students’ degree of agreement with five 
statements about their instruction: 

♦ I know what my teachers expect me to do; 

♦ I think of things not related to the lesson (reverse coded); 

♦ My teacher is easy to understand; 

♦ I am interested in what my teacher says; and 

♦ My teachers gives my interesting things to do. 

Students also reported about their reading lessons using the same scale but 
beginning with two questions specific to their reading materials. 

Students Engaged in Reading Lessons— Students’ degree of agreement 
with seven statements about their instruction: 

♦ I like what I read about in school; 

♦ My teacher gives me interesting things to read; 

♦ I know what my teachers expect me to do; 

♦ I think of things not related to the lesson (reverse coded); 

♦ My teacher is easy to understand; 

♦ I am interested in what my teacher says; and 

♦ My teachers gives my interesting things to do. 

Examining the Effects of Student Home Environment 

Considerable research has shown that higher levels of school resources are 
associated with higher achievement. However, the relationship between school 
resources and student achievement is complicated. On one hand, as described 
earlier under the discussion of the adequacy of school facilities and instructional 
resources, a school can invest more money for such things as facilities, teachers’ 
salaries, equipment, and materials. On the other hand, a school can have a 
more socioeconomically advantaged student population that has access to 
more resources, for example, because of its location or because it competes for 
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students. The home backgrounds of students attending a school can be closely 
related to the learning environment of the school, with the two reinforcing 
each other and being strongly linked to academic achievement. Students from 
higher socioeconomic backgrounds are likely to be healthy and come to school 
better fed and clothed than their more disadvantaged counterparts. Students 
from home backgrounds supportive of learning are likely to have more positive 
attitudes toward learning, and perhaps, even better discipline. They are likely 
to come to school already having the prerequisite literacy and numeracy skills 
necessary for advancing in the curriculum. Beyond that, parents that have high 
educational expectations for their children are more likely to take an active 
interest in the quality of teachers, the adequacy of school facilities, and the 
availability of school resources. 

In actuality, it is impossible to disentangle the effect of students’ home 
environment on their educational achievement from the effect of their schooling 
on their educational achievement. However, it is possible to apply statistical 
models to the data that make predictions about the likely effect of the school 
variables on student achievement if all students came from equivalent home 
backgrounds. 

Description of the Home Background Variables 

Two home background context questionnaire scales from TIMSS and PIRLS 
2011 were used to describe students’ home environment: 

♦ Home Resources for Learning Scale; and 

♦ Could Accomplish Early Literacy/Numeracy Tasks When Entered 
School. 

The Home Resources for Learning Scale is based on five different 
questions: two questions included in the student questionnaire, and three 
questions included in the home questionnaire completed by students’ parents 
and primary caregivers. Students were scored according to their own and their 
parents’ responses concerning the availability of the five resources listed below. 

Number of books in the home (students’ responses) 

1 . 0-10 

2. 11-25 

3. 26-100 

4. 101-200 

5. More than 200 
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Number of home study supports (students’ responses) 

1. None 

2. Internet connection or own room 

3. Both Internet and own room 

Number of childrens books in the home (parents’ responses) 

1 . 0-10 

2. 11-25 

3. 26-50 

4. 51-100 

5. More than 100 

Highest level of education of either parent (parents’ responses) 

1. Finished some primary or lower secondary or did not go to school 

2. Finished lower secondary 

3. Finished upper secondary 

4. Finished post-secondary education 

5. Finished university or higher 

Highest level of occupation of either parent (parents’ responses) 

1. Has never worked outside the home for pay, general laborer, or 
semi-professional (skilled agricultural or fishery worker, craft or 
trade worker, plant or machine operator) 

2. Clerical (clerk or service or sales worker) 

3. Small business owner 

4. Professional (corporate manger or senior official, professional, or 
technician or associate professional) 

Early Literacy/Numeracy Tasks was the student’s average of their scores 
on two early learning scales: Could Do Early Literacy Tasks When Began 
Primary School, and Could Do Early Numeracy Tasks When Began Primary 
School. 

Could Do Early Literacy Tasks When Began Primary School — Students 
were scored according to the parents’ responses to how well (i.e., “Very Well,” 
“Moderately Well,” “Not Very Well,” or “Not at All”) their child could do five 
early literacy activities when he/she began primary/elementary school: 
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♦ Recognize most of the letters of the alphabet; 

♦ Read some words; 

♦ Read sentences; 

♦ Write letters of the alphabet; and 

♦ Write some words. 

Could Do Early Numeracy Tasks When Began Primary School — Students 
were scored according to their parents’ responses to how well their child could 
do six early numeracy tasks, as shown below. 

Could your child do the following when he/she began primary/elementary 

school? 

1. Count by himself/herself (Up to 100 or higher, Up to 20, Up to 10, 
Not at all) 

2. Recognize different shapes (e.g., square, rectangle, circle) 

3. (More than 4 shapes, 3-4 shapes, 1-2 shapes, None) 

4. Recognize the written numbers from 1-10 (All 10 numbers, 5-9 
numbers, 1-4 numbers, None) 

5. Write the numbers from 1-10 (All 10 numbers, 5-9 numbers, 1-4 
numbers, None) 

6. Do simple addition (yes or no) 

7. Do simple subtraction (yes or no) 

The School Effectiveness Analysis 

In building an analytical model that shows the relationship between school 
variables and student achievement while controlling for the effects of home 
environment, it is important to recognize that these effects can operate at two 
levels: at the individual level through the direct effect of home environment 
on achievement, and at the school level through the effect of attending 
school with other students from similar advantaged or disadvantaged home 
backgrounds. Recognizing that, in any given school, students vary in their home 
backgrounds and also that schools can vary in the composition of their student 
body, this study adopted a two-level approach to statistically adjusting for 
home background differences, whereby both the differences between students 
within each school and the average differences between schools were explicitly 
modeled. This analysis shows the predicted effect on student achievement in 
reading, mathematics, and science of simultaneously adjusting the data so that 


EFFECTIVE SCHOOLS IN READING, 
TIMSS & PIRLS MATHEMATICS, AND SCIENCE AT THE FOURTH GRADE 

International Study Center 

Lynch School of Education, Boston College v H A r I 1 1\ O 


123 



all students in each school have equivalent home backgrounds and all schools 
have equivalent average home background. Although the analysis for each 
subject was based on a single model that controlled for both student-within- 
school and between-school differences, the results are presented separately so 
that the relative magnitude of their effects can be examined. 

To investigate how the characteristics of effective schools were associated 
with achievement in reading, mathematics, and science, this study made use of 
multilevel regression modeling (also known as hierarchical linear modeling). 
This type of prediction modeling allows characteristics of persons (such as 
students) and groups (such as schools) to be included together to predict 
individual-level outcomes, while accounting for the clustering of individuals 
in groups and maintaining correct standard errors for testing the significance 
of the relationships (Raudenbush & Bryk, 2002). The goal of the analyses was 
to examine how the characteristics of effective schools were associated with 
achievement in reading, mathematics, and science across countries, and whether 
schools in some countries were relatively more effective in one or two of these 
subjects than the others. 

Separately for each country, and for reading, mathematics, and science 
within each country, a series of multilevel regression models was formulated, 
each comprising a combination of the school explanatory measures and the 
student and school control variables. These models were used to describe 
how the school explanatory measures were associated with achievement, both 
before and after controlling for home background at student and school level. 
Specifically, for reading, mathematics and science, the models were grouped 
into three blocks, as described below. 

School Explanatory Models 

These models included the School Environment and School Instruction 
measures, separately and together, and were formulated in order to investigate 
the relationship between the school explanatory variables and student 
achievement in reading, mathematics, and science without reference to student 
home environment. 

Home Background Control Model 

This model included the Home Resources and Early Literacy/Numeracy 
measures together, and was formulated to investigate the relationship between 
home environment at the student and school level and student achievement in 
reading, mathematics, and science. 
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School Explanatory with Control Models 

Combining the School Explanatory models and the Home Background Control 
model, these models were formulated to investigate the relationship between the 
school explanatory variables, separately and together, and student achievement 
in reading, mathematics, and science, after controlling for the home background 
characteristics of individual students within the school and of the student body 
of the school. 

These multilevel regression models provided the detailed information 
needed to conduct a school effectiveness analysis of student achievement in 
reading, mathematics, and science. 

Interpreting the Multilevel Regression Models 

The regression coefficients in the multilevel models show the estimated effect 
of each predictor (school or student) variable, and are interpreted in the same 
way as an ordinary least squares regression coefficient; that is, for every one unit 
increase in the predictor variable, the outcome variable (student achievement) 
is predicted to increase or decrease by an amount indicated by the size and 
direction of the associated regression coefficient, holding all else constant. The 
regression coefficients are in the metric of the TIMSS and PIRLS achievement 
scales, in which 100 scale score points corresponds approximately to one 
standard deviation within a country. 2 The magnitude and direction of the 
regression coefficients in the models and the significance of the difference from 
zero indicate the relationship between each predictor and achievement, holding 
all else in the model constant. 

The percentage of variance explained by the predictors in the models is 
a useful summary of the strength of the relationship between the predictors 
and achievement in reading, mathematics, and science. The percentage of 
variance explained can be interpreted as the extent to which the variance in 
student achievement would be reduced if the data were adjusted so that all 
students had the same value on the predictor variable. For each country and 
separately for reading, mathematics, and science, the total student variance is 
decomposed into the percentages due to differences between schools (i.e., the 
extent to which schools differ in the average achievement of their students) 
and the differences between students within the schools. In the multilevel 
regression models, school-level predictors were added to explain school-to- 


2 The TIMSS achievement scales were established by TIMSS 1995, the first TIMSS assessment, so that 100 scale score 
points was equal to one standard deviation across all participating countries, and the scale centerpoint of 500 was 
equal to the mean score across countries. Scales were established separately by grade and for mathematics and science. 
Subsequently, data from TIMSS 1999, TIMSS 2003, TIMSS 2007, and TIMSS 2011 were placed on the TIMSS scale. Similarly, 
the PIRLS achievement scale was constructed so that 100 points was equal to the standard deviation across all countries 
that participated in PIRLS 2001, the first PIRLS assessment, and the centerpoint of 500 was set to the mean across 
countries. 

EFFECTIVE SCHOOLS IN READING, 
TIMSS & PIRLS MATHEMATICS, AND SCIENCE AT THE FOURTH GRADE 

International Study Center 

Lynch School of Education, Boston College v H A r I 1 1\ O 


125 



school differences in achievement. The percentage of variance explained by 
the school-level predictors and by the school-level measures was combined 
as the percentage of total variance in achievement explained, information 
that is analogous to an R 2 statistic. A detailed description of the analysis 
procedures, the types of information provided by the models, and guidance 
on the interpretation of the findings are presented in Technical Appendix B: 
School Effectiveness Models and Analyses. 

Results 

The multilevel regression modeling was conducted separately for reading, 
mathematics, and science for each country. As a prelude to these analyses, 
Exhibits 3.2 through 3.4 present, for reading, mathematics, and science, 
respectively, the decomposition of total student achievement variance into 
the percentages due to differences between schools and the differences within 
schools, together with the school-level correlations of each of the five school 
explanatory variables with achievement. The school-level correlations represent 
the correlation between the school score on the school explanatory variables 
and average student achievement in the school. 

According to the conceptual model for this study, student achievement 
should be higher in schools that are safe and orderly, have strong support for 
academic success, have adequate environment and resources, have a rigorous 
curriculum as evidenced by an early emphasis on reading skills, and where 
students are engaged in their reading, mathematics, and science lessons. On 
the basis of this model, the school-level correlations between each of these 
variables and student achievement should be positive and substantial in each 
country, because they represent the basic relationship between each school 
variable and average school achievement, without any statistical controls or 
adjustments. In fact, as shown in the exhibits, the correlations varied quite a lot 
both across countries and across the five explanatory variables. The correlations 
for reading, mathematics, and science were similar, on average, across countries 
and variables, although there were differences among individual countries. 

Among the School Environment variables, school-level correlations were 
strongest for Schools Support Academic Success, with average correlations 
across countries of 0.34 to 0.35 for the three subjects. There was considerable 
variation within countries, however, with highest correlations in Botswana (0.61 
to 0.62 across reading, mathematics, and science) and the lowest correlations 
in Italy (0.04 to 0.10). Correlations for Schools are Safe and Orderly were next 
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highest, on average, ranging from 0.28 to 0.29 across the subjects, with the 
highest correlations in Australia (0.54 to 0.55) and the lowest in Poland (-0.07 
to 0.14). Correlations were lower for the third School Environment variable, 
Adequate Environment and Resources, averaging 0.09 to 0.10 across countries. 
Correlations were highest in Qatar (0.33 to 0.39) and lowest in Croatia and the 
Czech Republic (-0.12). 

With respect to the two School Instruction variables, school-level 
correlations were highest for Students Engaged in Reading, Mathematics, and 
Science Lessons, with an average correlation across countries of 0.15 to 0.16 
for the three subjects. Correlations were highest in Botswana (0.59 to 0.64) and 
lowest in Poland (-0.19 to -0.23). In general, the lowest school-level correlations 
were for the School Instruction variable Early Emphasis on Reading Skills, 
with average correlations across countries of 0.07 to 0.08 for the three subjects. 
Correlations were highest in Dubai, UAE (0.48 for each subject) and lowest in 
Portugal (-0.10 to -0.12). 

Country-by-Country Analyses 

The results for individual countries are presented in Exhibits 3.5 through 
3.41, with one exhibit for each country. The school explanatory and home 
background control variables are listed as rows in the upper part of each exhibit, 
and the regression coefficients for the School Explanatory models, the Home 
Background Control model, and the School Explanatory with Control models 
as columns intersecting the rows. Results are included for reading, mathematics, 
and science for each model. Each regression coefficient is presented together 
with its standard error and an indicator of whether it differs significantly from 
zero. The lower part of each exhibit shows the variance decomposition (between 
and within schools) for each of the multilevel regression models, separately for 
reading, mathematics, and science. 

As an example of how the results of the school effectiveness analysis may 
be interpreted, the results for Australia, the first in the individual country 
presentations (Exhibit 3.5), are described in some detail. The Australian data 
show evidence of considerable differences among schools in student achievement 
(about one fourth of the total student variance — similar to the average across all 
countries), as well as strong relationships between the school environment and 
instruction variables and student achievement, and so are ideal for discussing 
the interpretation of results. The results for Botswana (Exhibit 3.37) also are 
described in detail. Botswana is a good example of a country where student 
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achievement differs considerably from school to school, and where school 
factors are related to student achievement even after controlling for student 
home background — i.e., some schools may be considered to be more “effective” 
than others. 

School Explanatory Models — Australia 

As discussed earlier, according to the conceptual model underpinning the 
effective schools analysis, student achievement in reading, mathematics, and 
science should be higher in schools that are safe and orderly, support academic 
success, and have adequate environment and resources than in schools that 
are deficient in one or more of these areas. The data show support for this 
proposition for Australia, with school-level correlations of 0.54 to 0.55 across 
the subjects for Schools are Safe and Orderly, 0.43 to 0.44 for Schools Support 
Academic Success, and 0.28 for Adequate Environment and Resources (see 
Exhibits 3.2, 3.3, and 3.4). 

As shown in the lower part of Exhibit 3.5, there were considerable 
differences among schools in Australia in the achievement of their students, 
with 23 percent of the total variance in reading achievement and 28 percent of 
the total variance in mathematics and science achievement due to differences 
between schools and available to be explained by school-to-school differences 
in the explanatory variables. 

The School Environment model (the first column of data in Exhibit 3.5) 
shows for each subject the predicted relationship with achievement of each 
of the three school environment variables when combined in a single model. 
This model reflects not only the relationship between the school environment 
variables and achievement but also any correlations among the school 
environment variables. In the Australian data, as might be expected given the 
school-level correlations, being in a safe and orderly school was the strongest 
predictor of achievement in each of the three subjects, with regression 
coefficients of 20 to 21 points after controlling for the other two variables. Being 
in a school that supported academic success also was a significant predictor, 
with regression coefficients of 6 points in each subject. However, the model 
shows no effect for being in a school with adequate environment and resources. 
Presumably, when the model statistically adjusts the data so that the schools 
have the same degree of orderliness and same level of support for academic 
success, any variation in environment and resources is eliminated. The school 
environment variables explained between 41 and 43 percent of the school-to- 
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school differences in reading, mathematics, and science scores in Australia, or 
between 10 and 12 percent of the total variance, suggesting that removing all 
differences in school environment would reduce total achievement differences 
by this amount. 

In accordance with the conceptual model, the School Instruction model for 
Australia predicts that students in schools where the student body was engaged 
in their lessons would have higher reading, mathematics, and science scores 
than students in other schools. These school instruction variables explained 6 
percent of the school-to-school differences in reading, mathematics, and science 
scores, or between 1 and 2 percent of the total variance. However, the model 
shows no effect for school-to-school variations in early emphasis in reading 
skills, either because controlling for level of engagement also eliminates any 
differences, or because there is no variation between Australian schools when 
emphasis is placed on various reading skills and strategies. 

Although, in Australia, both the School Environment and the School 
Instruction models contained significant predictors of student achievement 
when considered separately, when these were combined in a single School 
Environment and Instruction model, the Students Engaged predictor no longer 
made an independent contribution, suggesting that the statistical adjustment 
made by the model to give schools the same degree of orderliness and level of 
support for academic success removes any differences in student engagement 
in lessons. Accordingly, the combined School Environment and School 
Instruction variables explained 44 percent of the school-to-school differences 
in reading achievement, and 43 percent of the school-to-school differences in 
mathematics and science achievement, only marginally more than the School 
Environment variables alone. Overall, the School Explanatory models accounted 
for between 10 and 12 percent of the total variance in reading, mathematics, 
and science scores. 

Home Background Control Model — Australia 

The Home Background Control model for Australia provides evidence of a 
strong relationship between students’ home environment and their achievement 
in reading, mathematics, and science, and that this relationship operates both at 
the school level (in terms of the average level of the home background variables 
in the school, i.e., the student body composition), and within school (in terms 
of the difference between an individual student’s home background and the 
average for the school). In Australia, the predicted effect of attending a school 
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where many of the students came from well- resourced homes is particularly 
strong (regression coefficients of 49 to 56 points across subjects), suggesting 
that schools differ considerably in the composition of the student body and that 
achievement in all three subjects is higher in schools with many students from 
advantaged home backgrounds and lower in schools with many students from 
disadvantaged home backgrounds. 

In addition to the school composition effect in Australia, the Home 
Background Control model predicts an extra benefit from having a level of 
home background above the average for the school (and, conversely, an extra 
disadvantage to having a home background level below the average for the 
school). This effect is represented by the students within school regression 
coefficients, which are positive for both Home Resources for Learning and Early 
Literacy/Numeracy Tasks for all three subjects. 

The home background control variables explained between 58 and 61 
percent of the school-to-school differences in Australia, and between 7 and 
10 percent of the student-to-student differences within schools in reading, 
mathematics, and science scores. Overall, the student and school home 
background control variables explained 19 percent of the total variability in 
reading scores, and 24 percent of the total variability in mathematics and 
science scores. 

School Explanatory with Control Models — Australia 

The School Explanatory with Control models show the predicted effect on the 
school explanatory variables of statistically eliminating all differences between 
schools in the average level of student home background and also eliminating 
all home background differences among the students within the schools. For 
Australia, eliminating the differences in home background between schools and 
students had the effect of reducing the regression coefficients for Schools Are 
Safe and Orderly by half, and reducing the Schools Support Academic Success 
coefficients to just above zero. Although the Schools Are Safe and Orderly 
coefficients were reduced, they remained substantial (8 to 1 1 score points), and 
because all differences due to home background have been eliminated, these 
coefficients may be interpreted as the effects of the school environment variables 
over and above all other factors. 

Combined, the school explanatory and control variables accounted 
for about two-thirds of the school-to-school differences in achievement in 
Australia (67% for reading and mathematics; and 69% for science), representing 
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an increase of 8 to 9 percent over the amount accounted for by the home 
background control variables alone. Overall, 21 percent of the total variability 
in reading scores, and 26 percent of the total variability in mathematics and 
science scores was explained by the school explanatory variables and the home 
background control variables. 

Effective Schools Analysis — Botswana 

As one of the countries with relatively large differences between schools in 
achievement, Botswana provides a further example of how the effective schools 
analysis can reveal differential effects of school environment and instructional 
variables on student achievement, even after adjusting for home background 
effects. As shown in Exhibit 3.37, about one-third of the total variance in student 
achievement (38% in reading, 31% in mathematics, and 35% in science) was due 
to differences between schools and was available to be explained by school-to- 
school differences in the explanatory variables. 

In Botswana, school-level correlations with achievement for the school 
environment variables were highest for Schools Support Academic Success 
(0.61 to 0.62), next highest for Schools Are Safe and Orderly (0.46 to 
0.49), and lowest for Adequate Environment and Resources (0.22 to 0.23) 
(see Exhibits 3.2, 3.3, and 3.4). When these were combined in the School 
Environment model (the first column of data in Exhibit 3.37), being in a school 
that supported academic success and being in a safe and orderly school were the 
strongest predictors of achievement in each of the three subjects, with regression 
coefficients of 14 to 22 points after controlling for each of the other variables. 
As for Australia, the model for Botswana showed no effect for being in a school 
with adequate environment and resources. The school environment variables 
explained between 44 and 46 percent of the school-to-school differences in 
reading, mathematics, and science scores in Botswana, or between 14 and 17 
percent of the total variance, suggesting that eliminating all differences in school 
environment would reduce total achievement differences by this amount. This 
was slightly more than in Australia. 

Although Botswana was similar to Australia in that there was essentially no 
correlation between Early Emphasis on Reading Skills and achievement, the 
countries differed in that there was a much stronger correlation with Students 
Engaged in their Lessons in Botswana— about 0.6, compared to about 0.2. The 
School Instruction model for Botswana accounted for 36 to 43 percent of the 
school-to-school differences in reading, mathematics, and science scores, or 
between 13 and 15 percent of the total variance. Further, in the combined School 
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Environment and School Instruction model, the Students Engaged in their 
Lessons variable had the strongest relationship with achievement (regression 
coefficients of 32 to 54 points), followed by Schools Support Academic Success 
(regression coefficients of 11 to 17 points). Altogether, in Botswana the School 
Environment and School Instruction variables accounted for 58 percent of the 
school-to-school differences in reading achievement, 62 percent in mathematics, 
and 63 percent in science, or between 19 and 22 percent of the total variance in 
reading, mathematics, and science scores. 

Similar to the situation in Australia, the Home Background Control model 
for Botswana showed a strong relationship between students’ home environment 
and their achievement in reading, mathematics, and science. The predicted 
effect of attending a school where many of the students came from well- 
resourced homes is particularly strong (regression coefficients 26 to 39 points), 
and the within-school effect, the predicted advantage or disadvantage of having 
a level of home background above or below the average for the school, also 
was significant for all three subjects. The home background control variables 
explained between 56 and 68 percent of the school-to-school differences, and 
between 4 and 7 percent of the student-to-student differences within schools 
in reading, mathematics, and science scores. Overall, the student and school 
home background control variables explained 30 percent of the total variability 
in reading scores, 20 percent in mathematics, and 25 percent in science. 

In Botswana, in contrast to Australia, eliminating the differences in home 
background between schools and students (the School Explanatory with Control 
model) does not remove the School Instruction effects, and reduces but does not 
eliminate the School Environment effects. The predicted effect of the Students 
Engaged in their Lessons variable was reduced somewhat but still substantial 
(regression coefficients of 27 to 44 points), while the regression coefficients for 
Schools Are Safe and Orderly (7 to 9 points) and Schools Support Academic 
Success (5 to 7 points) retain small but significant effects. 

Combined, the school explanatory and control variables accounted for 
four-fifths or more of the school-to-school differences in achievement in 
Botswana (84% for reading and science; and 80% for mathematics), representing 
an increase of 16 to 24 percentage points over the amount accounted for by 
the home background control variables alone. Overall, 36 percent of the total 
variability in reading scores, 28 percent in mathematics, and 33 percent in 
science was accounted for by the school explanatory variables and the home 
background control variables. 
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Results across TIMSS and PIRLS 20 1 1 Participants 

Looking across the individual country results in Exhibits 3.5 through 3.41, the 
average percentage of variance due to differences between schools was fairly 
similar for reading (22%), mathematics (26%), and science (25%), although 
there were considerable differences from country to country. Slovenia had 
the smallest percentage of variance in achievement between schools, with 5 
percent for reading and 8 percent for mathematics and science. In several other 
countries, including Austria, Chinese Taipei, Croatia, Finland, Norway, and 
Poland, the percentage of variance between schools was 10 percent or less in at 
least one subject. Because schools in these countries do not differ very much in 
the average achievement of their students, there is little scope for finding school 
variables that account for this difference. 

The largest percentage of variance in achievement between schools was 
observed for Dubai, UAE, where approximately 50 percent was between schools. 
Honduras, United Arab Emirates, Qatar, Azerbaijan, and Abu Dhabi, UAE also 
had relatively large percentages of variance between schools. 

The Home Background Control model was successful in capturing the 
relationship between home background and student achievement in reading, 
mathematics, and science in every participant, although the exact nature of the 
relationship varied among the countries. The Home Resources for Learning 
variable was the strongest predictor, with significant effects at both the school 
level (in terms of the average level of home resources for students in the school) 
and within the school (in terms of the difference between an individual’s 
home resources and the average for the school) in almost every country. The 
Early Literacy/Numeracy Tasks was a less powerful predictor, with significant 
between-school effects in about half of the countries. 

As shown in Exhibits 3.2 through 3.5, the school variables posited by the 
conceptual model are positively correlated with student achievement in most 
countries, providing prima facie evidence from the data for the validity of the 
model. The School Explanatory models show that many of these relationships 
persist when the school environment and school instruction variables are 
combined in a single model (without any other controls). Only two countries, 
the Czech Republic and the Russian Federation, had no significant predictors 
of student achievement in these School Explanatory models. 

The School Explanatory with Home Background Control model shows how 
the effect of the School Environment and Instruction variables on achievement 
is predicted to change when the data are adjusted statistically so that all students 
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have the same home background. As would be expected, given that schools with 
many students from supportive home environments often have positive school 
environment and instruction, introducing the home background variables as 
controls reduces the strength of the relationship between school environment 
and instruction and student achievement. Whereas only in the Czech Republic 
and the Russian Federation was there no relationship between the school 
variables and achievement before controlling for home background, there were 
seven more countries with no significant relationship after including the home 
background controls: Austria, Honduras, Iran, Poland, Romania, Slovenia, 
and Sweden. 

All of the 28 remaining countries and benchmarking participants had 
a significant relationship between at least one of the School Environment or 
Instruction variables and achievement in reading, mathematics, or science 
after controlling for home background. Of the School Environment variables, 
Schools Are Safe and Orderly was related to achievement in at least one subject 
over and above the effects of home background in 15 countries, and in all three 
subjects in 7 countries. Schools Support Academic Success was a positive 
predictor of achievement in at least one subject in 10 countries and in all three 
in 2 countries. Adequate Environment and Resources had a predicted effect 
on achievement independent of home background in at least one subject in just 
3 countries, and in all three subjects in just one country. 

Of the two School Instruction variables, Students Engaged in Reading, 
Mathematics, and Science Lessons was the more powerful predictor, and was 
positively related to achievement in at least one subject in 17 countries after 
controlling for home background and in all three subjects in 9 countries. In 
contrast, Early Emphasis in Reading Skills was a significant predictor in just 
two countries. 

Looking across the countries, 15 of the participants had just one significant 
predictor after controlling for home background. These included Australia, 
Chinese Taipei, Croatia, Finland, Lithuania, Northern Ireland, Norway, the 
Slovak Republic (Schools Are Safe and Orderly); Quebec (Schools Support 
Academic Success); Italy (Adequate Environment and Resources); and 
Georgia, Hong Kong SAR, Hungary, Portugal, and Singapore (Students 
Engaged in Reading, Mathematics, and Science Lessons). Students Engaged 
in Reading, Mathematics, and Science Lessons was a significant predictor after 
controlling for home background in all 8 of the countries with two significant 
predictors, with Schools Support Academic Success the second predictor in 
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Azerbaijan, Ireland, Saudi Arabia, Abu Dhabi, and Dubai; Schools Are Safe and 
Orderly the second predictor in Malta and Spain; and Adequate Environment 
and Resources the second predictor in Morocco. 

Germany, Oman, Qatar, and Botswana each had three significant predictors 
of achievement after controlling for home background, with Schools Are Safe 
and Orderly a significant predictor in each case. The other two predictors 
were Schools Support Academic Success and Students Engaged in Reading, 
Mathematics, and Science Lessons in Oman and Botswana; Schools Support 
Academic Success and Early Emphasis on Reading Skills in Germany; and 
Schools Support Academic Success and Students Engaged in Reading, 
Mathematics, and Science Lessons in Qatar. The largest number of significant 
predictors after controlling for home background was in the United Arab 
Emirates, where Schools Are Safe and Orderly, Schools Support Academic 
Success, Early Emphasis on Reading Skills, and Students Engaged in Reading, 
Mathematics, and Science Lessons each had a positive effect. 

Summary 

Conscious that education systems typically have evolved as the result of 
management and development over many years, and that the variables of interest 
have been manipulated to achieve policy goals so that expected relationships 
may not be apparent in the data, this study relied on a strong conceptual model 
based on a clear vision of the essential characteristics of effective schools to 
guide the analyses. Building on the school effectiveness research literature and 
capitalizing on the unique array of school and student variables available in the 
TIMSS and PIRLS 2011 Fourth Grade Combined International Database (Foy, 
2013), the conceptual model specified a small number of school characteristics 
considered essential for effective schooling in all countries: a school environment 
that was safe and orderly, supportive of academic success, and with adequate 
facilities and equipment, and school instruction that emphasized higher order 
reading processes and student engagement in reading, mathematics, and 
science lessons. 

Because the conceptual model is based on the idea that all effective schools 
possess these characteristics to some degree, it raises the expectation that 
the TIMSS and PIRLS data would show positive relationships between these 
characteristics and student achievement in each country. In fact, however, the 
countries varied considerably in the extent to which the relationships predicted 
by the conceptual model were observed in the data. In a number of countries, 
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most notably Slovenia, Austria, Chinese Taipei, Croatia, Finland, Norway, and 
Poland, primary schools are organized so that there is little school-to-school 
difference in student achievement and, consequently, there is little scope for 
relationships between school characteristics and achievement at the fourth grade 
in the data from these countries. School effectiveness analyses in countries such 
as these are limited in the information they can provide about characteristics 
of effective schooling. 

However, there also were countries where schools differed considerably 
in the achievement of their students, to the extent that, on average across all 
participating countries, about one fourth of the total student variance was 
attributable to differences among schools. In such countries, a strong positive 
relationship often was in evidence between one or more school characteristic 
and achievement in reading, mathematics, and science. For example, Botswana, 
as described earlier, had considerable differences among schools in student 
achievement and positive relationships between school environment and 
instruction, home background, and student achievement, and these relationships 
persisted after controlling for the effects of home background. 

In summary, this study found considerable differences across countries 
in the way student achievement is distributed across schools and in the way 
school variables are related to student achievement, although the results were 
fairly similar for reading, mathematics, and science. For example, the average 
percentage of variance due to differences between schools was 22 percent for 
reading, 26 percent for mathematics, and 25 percent for science. Despite the 
differences, the Home Background Control model was successful in capturing 
the relationship between home background and student achievement in reading, 
mathematics, and science in every country, although the exact nature of the 
relationship varied among the countries. The Home Resources for Learning 
variable was the strongest predictor, with significant effects at both the school 
level and within the school in almost every country. The Early Literacy/ 
Numeracy Tasks was a less powerful predictor, with significant between-school 
effects in about half of the countries. 
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The school variables posited by the conceptual model were positively 
correlated with student achievement in most countries, providing prima facie 
evidence from the data for the validity of the model. The School Explanatory 
models showed that many of these relationships persisted when the school 
environment and school instruction variables were combined in a single model 
(without any other controls). Only two countries, the Czech Republic and the 
Russian Federation, had no significant predictors of student achievement in 
these School Explanatory models. Almost all of the remaining countries and 
benchmarking participants had a significant relationship between at least one of 
the School Environment or Instruction variables and achievement in reading, 
mathematics, or science after controlling for home background. 

Of the School Environment variables, Schools Are Safe and Orderly 
was related to achievement in at least one subject over and above the effects 
of home background in 15 countries, and in all three subjects in 7 countries. 
Schools Support Academic Success was a positive predictor of achievement 
in at least one subject in 10 countries and in all three in 2 countries. Adequate 
Environment and Resources had a predicted effect on achievement 
independent of home background in at least one subject in just 3 countries, 
and in all three subjects in just one country. Of the two School Instruction 
variables, Students Engaged in Reading, Mathematics, and Science Lessons 
was the more powerful predictor, and was positively related to achievement in 
at least one subject in 17 countries after controlling for home background and in 
all three subjects in 9 countries. In contrast, Early Emphasis in Reading Skills 
was a significant predictor in just two countries. 
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Exhibit 3.2: Variance Decomposition and Correlations of School Explanatory Variables with TIMSS &PIRLS 
Reading Achievement 2011 


4 th 

Grade 


Country 

Percentage of 

School-level Correlations 

Total Variance 

Schools 

Schools 

Adequate 

Early 

Students Engaged 


Between 

Schools 

Within 

Schools 

Are Safe 
and Orderly 

Support Academic 
Success 

Environment 
and Resources 

Emphasis on 
Reading Skills 

in Reading, 
Mathematics, and 
Science Lessons 

Australia 

23 

77 

0.55 

0.44 

0.28 

0.03 

0.21 

Austria 

9 

91 

0.36 

0.36 

0.04 

-0.05 

-0.13 

Azerbaijan 

44 

56 

0.23 

0.16 

0.05 

-0.03 

0.43 

Chinese Taipei 

10 

90 

0.18 

0.27 

0.03 

-0.03 

0.27 

Croatia 

10 

90 

0.09 

0.22 

-0.12 

0.11 

-0.14 

Czech Republic 

15 

85 

0.04 

0.15 

-0.12 

-0.03 

-0.01 

Finland 

7 

93 

0.39 

0.40 

0.08 

0.13 

-0.06 

Georgia 

25 

75 

0.12 

0.22 

-0.03 

0.01 

0.37 

Germany 

24 

76 

0.43 

0.53 

0.16 

0.24 

-0.04 

Hong Kong SAR 

21 

79 

0.17 

0.04 

-0.03 

0.19 

0.40 

Hungary 

32 

68 

0.45 

0.52 

-0.03 

0.07 

0.02 

Iran, Islamic Rep. of 

39 

61 

0.19 

0.35 

0.13 

0.25 

0.00 

Ireland 

12 

88 

0.55 

0.43 

0.13 

0.12 

0.07 

Italy 

17 

83 

0.20 

0.04 

0.15 

-0.02 

0.08 

Lithuania 

19 

81 

0.24 

0.40 

0.00 

0.15 

0.20 

Malta 

28 

72 

0.45 

0.55 

0.26 

0.03 

0.47 

Morocco 

38 

62 

0.25 

0.45 

0.21 

0.28 

0.45 

Northern Ireland 

11 

89 

0.41 

0.34 

0.11 

0.08 

0.18 

Norway 

8 

92 

0.31 

0.36 

-0.01 

-0.02 

0.24 

Oman 

18 

82 

0.20 

0.41 

0.24 

0.26 

0.34 

Poland 

10 

90 

-0.07 

0.35 

-0.01 

-0.04 

-0.19 

Portugal 

18 

82 

0.26 

0.40 

-0.01 

-0.12 

0.37 

Qatar 

38 

62 

0.52 

0.43 

0.36 

0.23 

0.45 

Romania 

35 

65 

0.35 

0.44 

0.08 

0.13 

0.37 

Russian Federation 

25 

75 

0.15 

0.20 

0.14 

-0.03 

0.03 

Saudi Arabia 

36 

64 

0.31 

0.54 

0.26 

0.18 

0.48 

Singapore 

25 

75 

0.29 

0.43 

-0.01 

-0.02 

0.08 

Slovak Republic 

18 

82 

0.27 

0.39 

0.04 

0.04 

-0.03 

Slovenia 

5 

95 

0.09 

0.17 

0.03 

-0.04 

-0.07 

Spain 

18 

82 

0.36 

0.42 

0.03 

-0.11 

0.18 

Sweden 

13 

87 

0.47 

0.36 

0.25 

0.08 

-0.07 

United Arab Emirates 

43 

57 

0.39 

0.39 

0.33 

0.42 

0.32 

International Avg. 

22 1 

78 

0.29 

0.35 

0.09 

0.08 

0.16 jl 

Sixth Grade Countries 

Botswana 

38 

62 

0.46 

0.62 

0.22 

0.02 

0.59 

Honduras 

43 

57 

0.15 

0.10 

0.35 

0.25 

-0.06 

Benchmarking Participants 

Quebec, Canada 

11 

89 

0.30 

0.45 

0.26 

0.02 

0.21 

Abu Dhabi, UAE 

40 

60 

0.36 

0.41 

0.33 

0.21 

0.30 

Dubai, UAE 

51 

49 

0.43 

0.38 

0.34 

0.48 

0.39 
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Exhibit 3.3: Variance Decomposition and Correlations of School Explanatory Variables with 
Mathematics Achievement 


TIMSS fr PIRLS 
2011 


^th 

Grade 


Country 

Percentage of 


School-level Correlations 


Total Variance 

Schools 

Schools 

Adequate 

Early 

Students Engaged 


Between 

Schools 

Within 

Schools 

Are Safe 
and Orderly 

Support Academic 
Success 

Environment 
and Resources 

Emphasis on 
Reading Skills 

in Reading, 
Mathematics, and 
Science Lessons 

Australia 

28 

72 

0.54 

0.43 

0.28 

0.04 

0.21 

Austria 

16 

84 

0.38 

0.37 

0.15 

-0.05 

-0.08 

Azerbaijan 

48 

52 

0.19 

0.16 

0.11 

-0.08 

0.54 

Chinese Taipei 

11 

89 

0.18 

0.27 

0.06 

-0.01 

0.26 

Croatia 

13 

87 

0.06 

0.21 

-0.02 

0.16 

-0.13 

Czech Republic 

21 

79 

0.04 

0.14 

-0.12 

-0.01 

0.00 

Finland 

9 

91 

0.39 

0.41 

0.14 

0.18 

-0.06 

Georgia 

38 

62 

0.07 

0.25 

-0.03 

-0.04 

0.34 

Germany 

27 

73 

0.39 

0.52 

0.13 

0.21 

-0.08 

Hong Kong SAR 

22 

78 

0.18 

0.13 

-0.03 

0.18 

0.41 

Hungary 

36 

64 

0.48 

0.53 

-0.02 

0.07 

0.01 

Iran, Islamic Rep. of 

38 

62 

0.15 

0.31 

0.11 

0.20 

-0.02 

Ireland 

17 

83 

0.52 

0.42 

0.12 

0.07 

0.12 

Italy 

26 

74 

0.18 

0.10 

0.20 

0.01 

0.08 

Lithuania 

20 

80 

0.21 

0.42 

0.00 

0.13 

0.14 

Malta 

20 

80 

0.47 

0.54 

0.23 

0.00 

0.41 

Morocco 

44 

56 

0.21 

0.36 

0.20 

0.20 

0.41 

Northern Ireland 

17 

83 

0.42 

0.28 

0.07 

0.03 

0.20 

Norway 

14 

86 

0.35 

0.35 

0.04 

0.02 

0.21 

Oman 

20 

80 

0.18 

0.36 

0.16 

0.23 

0.35 

Poland 

13 

87 

-0.11 

0.35 

-0.03 

-0.04 

-0.23 

Portugal 

38 

62 

0.28 

0.40 

0.04 

-0.11 

0.35 

Qatar 

45 

55 

0.44 

0.38 

0.39 

0.22 

0.36 

Romania 

38 

62 

0.37 

0.43 

0.09 

0.09 

0.29 

Russian Federation 

34 

66 

0.16 

0.12 

0.09 

-0.05 

0.05 

Saudi Arabia 

37 

63 

0.21 

0.37 

0.12 

0.09 

0.22 

Singapore 

25 

75 

0.27 

0.42 

-0.03 

-0.01 

0.10 

Slovak Republic 

27 

73 

0.29 

0.38 

0.02 

0.06 

0.04 

Slovenia 

8 

92 

0.08 

0.17 

0.00 

-0.06 

-0.11 

Spain 

21 

79 

0.42 

0.49 

0.03 

-0.05 

0.20 

Sweden 

12 

88 

0.51 

0.46 

0.28 

0.14 

-0.09 

United Arab Emirates 

45 

55 

0.34 

0.36 

0.30 

0.42 

0.27 

International Avg. 

26 

74 

0.28 

0.34 | 

0.10 1 

0.07 

0.15 

Sixth Grade Countries 

Botswana 

31 

69 

0.49 

0.61 

0.23 

0.03 

0.62 

Honduras 

47 

53 

0.13 

0.13 

0.39 

0.21 

-0.04 

Benchmarking Participants 

Quebec, Canada 

15 

85 

0.29 

0.45 

0.27 

-0.05 

0.08 

Abu Dhabi, UAE 

42 

58 

0.33 

0.37 

0.31 

0.25 

0.22 

Dubai, UAE 

52 

48 

0.41 

0.37 

0.33 

0.48 

0.36 
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Exhibit 3.4: Variance Decomposition and Correlations of School Explanatory Variables with TIMSS &PIRLS 
Science Achievement 2011 


4.U1 

Grade 


Country 

Percentage of 

School-level Correlations 

Total Variance 

Schools 

Schools 

Adequate 

Early 

Students Engaged 


Between 

Schools 

Within 

Schools 

Are Safe 
and Orderly 

Support Academic 
Success 

Environment 
and Resources 

Emphasis on 
Reading Skills 

in Reading, 
Mathematics, and 
Science Lessons 

Australia 

28 

72 

0.54 

0.44 

0.28 

0.02 

0.21 

Austria 

13 

87 

0.42 

0.38 

0.14 

-0.04 

-0.15 

Azerbaijan 

49 

51 

0.22 

0.18 

0.06 

-0.04 

0.55 

Chinese Taipei 

10 

90 

0.18 

0.26 

0.06 

0.01 

0.24 

Croatia 

11 

89 

0.10 

0.20 

-0.12 

0.10 

-0.13 

Czech Republic 

14 

86 

0.07 

0.15 

- 0.10 

-0.03 

0.02 

Finland 

10 

90 

0.42 

0.43 

0.08 

0.14 

-0.13 

Georgia 

32 

68 

0.05 

0.21 

-0.07 

-0.03 

0.34 

Germany 

26 

74 

0.44 

0.55 

0.16 

0.20 

-0.07 

Hong Kong SAR 

20 

80 

0.13 

0.08 

-0.04 

0.18 

0.36 

Hungary 

38 

62 

0.46 

0.51 

-0.04 

0.06 

0.01 

Iran, Islamic Rep. of 

42 

58 

0.13 

0.32 

0.09 

0.20 

-0.06 

Ireland 

19 

81 

0.53 

0.42 

0.12 

0.12 

0.12 

Italy 

26 

74 

0.20 

0.07 

0.15 

- 0.01 

0.06 

Lithuania 

23 

77 

0.23 

0.38 

- 0.01 

0.12 

0.18 

Malta 

24 

76 

0.41 

0.55 

0.31 

0.01 

0.41 

Morocco 

36 

64 

0.22 

0.40 

0.21 

0.22 

0.45 

Northern Ireland 

22 

78 

0.40 

0.31 

0.05 

0.05 

0.21 

Norway 

9 

91 

0.34 

0.41 

0.03 

-0.03 

0.20 

Oman 

21 

79 

0.19 

0.36 

0.15 

0.23 

0.40 

Poland 

11 

89 

-0.13 

0.34 

-0.05 

-0.06 

-0.23 

Portugal 

34 

66 

0.26 

0.42 

0.02 

- 0.10 

0.37 

Qatar 

42 

58 

0.48 

0.39 

0.33 

0.18 

0.41 

Romania 

37 

63 

0.38 

0.42 

0.08 

0.13 

0.35 

Russian Federation 

33 

67 

0.16 

0.15 

0.12 

- 0.01 

0.05 

Saudi Arabia 

37 

63 

0.29 

0.50 

0.21 

0.15 

0.39 

Singapore 

25 

75 

0.27 

0.44 

-0.02 

-0.02 

0.06 

Slovak Republic 

26 

74 

0.28 

0.37 

0.02 

0.05 

0.03 

Slovenia 

8 

92 

0.09 

0.14 

0.01 

-0.05 

- 0.10 

Spain 

19 

81 

0.37 

0.43 

0.04 

-0.06 

0.16 

Sweden 

15 

85 

0.54 

0.41 

0.27 

0.08 

- 0.10 

United Arab Emirates 

41 

59 

0.36 

0.39 

0.30 

0.39 

0.35 

International Avg. 

25 

75 

0.28 

| 0.34 

0.09 1 

0.07 

0.16 ; 

Sixth Grade Countries 

Botswana 

35 

65 

0.47 

0.61 

0.22 

0.03 

0.64 

Honduras 

52 

48 

0.13 

0.10 

0.32 

0.22 

-0.09 

Benchmarking Participants 

Quebec, Canada 

14 

86 

0.28 

0.45 

0.22 

-0.03 

0.17 

Abu Dhabi, UAE 

38 

62 

0.35 

0.41 

0.31 

0.20 

0.32 

Dubai, UAE 

49 

51 

0.42 

0.39 

0.33 

0.48 

0.41 
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TI2VI.SS PJRLSI 

Exhibit 3.5: School Effectiveness Models for Reading, Mathematics, and Science - Australia 2011 



HLM Regression Coefficients 


School Explanatory Models 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 

School Environment 



REA 

21 (3.5) 

o 

— 

20 (3.5) 

o 

— 


10 (3.5) 

o 

— 


10 (3.5) 

o 

Schools Are Safe and Orderly 

MAT 

21 (4.1) 

o 

— 

21 (4.1) 

o 

— 


11 (4.0) 

o 

— 


11 (4.0) 

o 


SCI 

20 (3.5) 

o 

— 

19 (3.5) 

o 

— 


8 (3.3) 

o 

— 


8 (3.3) 

o 


REA 

6 (2.1) 

o 

— 

6 (2.1) 

o 

— 


2 (1.8) 


— 


3 (1.8) 


Schools Support Academic Success 

MAT 

6 (2.4) 

o 

— 

7 (2.4) 

o 

— 


3 (2.1) 


— 


3 (2.1) 



SCI 

6 (2.0) 

o 

— 

6 (2.0) 

o 

— 


3 (1.6) 


— 


3 (1.6) 



REA 

2 (2.5) 


— 

2 (2.5) 


— 


1 (1.9) 


— 


1 (1.9) 


Adequate Environment and Resources 

MAT 

3 (2.5) 


— 

3 (2.5) 


— 


2 (2.0) 


— 


1 (2.0) 



SCI 

2 (2.5) 


— 

2 (2.5) 


— 


1 (1.8) 


— 


1 (1.8) 


School Instruction 


REA 

— 


0 (1.8) 

2 (1.5) 


— 


— 


1 (1.4) 


1 (1.3) 


Early Emphasis on Reading Skills 

MAT 

— 


1 (1.9) 

2 (1.7) 


— 


— 


2 (1.5) 


2 (1.4) 



SCI 

— 


0 (1.7) 

2 (1.5) 


— 


— 


1 (1.3) 


1 (1.2) 


Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 


15 (5.2) O 

6 (4.2) 


— 


— 


7 (3.7) 


4 (3.6) 


MAT 

— 


16 (5.6) O 

6 (4.3) 


— 


— 


6 (3.7) 


3 (3.6) 


SCI 

— 


14(5.2) O 

5 (4.2) 


— 


— 


6 (3.2) 


3 (3.2) 


Home Background Control Variables 

Students within Schools 


REA 

— 


— 

— 


12 (1.3) 

o 

12 (1.3) 

o 

12 (1.3) 

o 

12 (1.3) 

o 

Home Resources for Learning 

MAT 

— 


— 

— 


12 (1.4) 

o 

12 (1.4) 

o 

12 (1.4) 

o 

12 (1.4) 

o 


SCI 

— 


— 

— 


13 (1.1) 

o 

13 (1.1) 

o 

13 (1.1) 

o 

13 (1.1) 

o 


REA 

— 


— 

— 


11 (1.3) 

o 

11 (1.3) 

o 

11 (1.3) 

o 

11 (1.3) 

o 

Early Literacy/Numeracy Tasks 

MAT 

— 


— 

— 


15 (1.4) 

o 

15 (1.4) 

o 

15 (1.4) 

o 

15 (1.4) 

o 


SCI 

— 


— 

— 


12 (1.2) 

o 

12 (1.2) 

o 

12 (1.2) 

o 

12 (1.2) 

o 

Between Schools 

School Average of 
Home Resources for Learning 

REA 

— 


— 

— 


49 (3.7) 

o 

38 (3.8) 

o 

48 (3.9) 

o 

38 (3.8) 

o 

MAT 

— 


— 

— 


56 (4.0) 

o 

43 (3.9) 

o 

55 (3.9) 

o 

42 (3.9) 

o 

SCI 

— 


— 

— 


49 (3.4) 

o 

40 (3.6) 

o 

48 (3.5) 

o 

39 (3.6) 

o 

School Average of 

Early Literacy/Numeracy Tasks 

REA 

— 


— 

— 


19 (6.8) 

o 

15 (7.0) 

o 

17 (6.9) 

o 

15 (7.0) 

o 

MAT 

— 


— 

— 


23 (8.4) 

o 

19 (8.8) 

o 

22 (8.2) 

o 

19 (8.5) 

o 

SCI 

— 


— 

— 


21 (6.0) 

o 

18 (6.2) 

o 

19 (6.2) 

o 

17 (6.2) 

o 


() Standard errors appear in parentheses. REA -Reading O Coefficient significantly greaterthan zero. 


MAT 

SCI 

-Mathematics ® Coefficient significantly less than zero. 

-Science 







Percentage of Variance Explained 

Source of Variance 


School Explanatory Models 


Home 


School Explanatory with Home Background Control Models 



School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


Background 
Control Model 


School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Reading 








Between Schools (23%) 

43 

6 

44 

58 

66 

59 

67 

Within Schools (77%) 

— 

— 

— 

7 

6 

7 

6 

Total 

10 

1 

10 

19 

20 

19 

21 

Mathematics 








Between Schools (28%) 

41 

6 

43 

58 

66 

60 

67 

Within Schools (72%) 

— 

— 

— 

10 

10 

10 

10 

Total 

12 

2 

12 

24 

26 

24 

26 

Science 








Between Schools (28%) 

42 

6 

43 

61 

68 

62 

69 

Within Schools (72%) 

— 

— 

— 

10 

10 

10 

10 

Total 

12 

2 

12 

24 

26 

25 

26 


( ) Percentage of available variance shown in parentheses. 
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Exhibit 3.6: School Effectiveness Models for Reading, Mathematics, and Science - Austria 


TIMSS & PIRLS 
2011 



HLM Regression Coefficients 


School Explanatory Models 1; 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 

School Environment 



REA 

8 (3.3) 

o 

— 

8 (3.3) 

o 

— 


4 (2.4) 

— 


4 (2.4) 


Schools Are Safe and Orderly 

MAT 

9 (4.1) 

o 

— 

9 (4.1) 

o 

— 


6 (3.7) 

— 


5 (3.6) 



SCI 

10 (3.7) 

o 

— 

10 (3.7) 

o 

— 


6 (3.1) 

— 


5 (3.0) 



REA 

5 (2.4) 

o 

— 

5 (2.4) 

o 

— 


2 (2.0) 

— 


2 (1.9) 


Schools Support Academic Success 

MAT 

5 (2.6) 

o 

— 

6 (2.6) 

o 

— 


2 (2.4) 

— 


3 (2.4) 



SCI 

6 (2.4) 

o 

— 

6 (2.5) 

o 

— 


2 (2.2) 

— 


2 (2.2) 



REA 

-2 (1.3) 


— 

-2 (1.4) 


— 


-2 (1.2) 

— 


-2 (1.2) 


Adequate Environment and Resources 

MAT 

0 (1.7) 


— 

0 (1.7) 


— 


0 (1.6) 

— 


1 (1.6) 



SCI 

-1 (1.5) 


— 

-1 (1.6) 


— 


0 (1.3) 

— 


0 (1.3) 


School Instruction 


REA 

— 


-1 (1.7) 

-2 (1.6) 


— 


— 

-2 (1.5) 


-2 (1.4) 


Early Emphasis on Reading Skills 

MAT 

— 


-2 (2.0) 

-3 (1.7) 


— 


— 

-3 (1.8) 


-3 (1.6) 



SCI 

— 


-1 (2.0) 

-2 (1.7) 


— 


— 

-3 (1.8) 


-3 (1.6) 


Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 


-4 (3.9) 

-4 (3.8) 


— 


— 

-2 (3.2) 


-2 (3.2) 


MAT 

— 


-3 (5.1) 

-2 (4.9) 


— 


— 

0 (4.5) 


1 (4.6) 


SCI 

— 


-5 (4.9) 

-5 (4.7) 


— 


— 

-2 (3.9) 


-2 (4.0) 


Home Background Control Variables 

Students within Schools 


REA 

— 


— 

— 


19 (0.8) 

o 

19 (0.8) O 

19 (0.8) 

o 

19 (0.8) 

o 

Home Resources for Learning 

MAT 

— 


— 

— 


16 (0.7) 

o 

16 (0.7) O 

16 (0.7) 

o 

16 (0.7) 

o 


SCI 

— 


— 

— 


20 (0.9) 

o 

20 (1.0) O 

20 (1.0) 

o 

20 (1.0) 

o 


REA 

— 


— 

— 


5 (0.8) 

o 

5(0.8) O 

5 (0.8) 

o 

5 (0.8) 

o 

Early Literacy/Numeracy Tasks 

MAT 

— 


— 

— 


8 (0.7) 

o 

8(0.7) O 

8 (0.7) 

o 

8 (0.7) 

o 


SCI 

— 


— 

— 


5 (0.7) 

o 

5(0.7) O 

5 (0.7) 

o 

5 (0.7) 

o 


Between Schools 

School Average of 
Home Resources for Learning 


School Average of 

Early Literacy/Numeracy Tasks 

( ) Standard errors appear in parentheses. 


REA 

— 

— 

— 

27 (2.6) O 

24 (2.9) O 

27 (2.6) O 

24 (2.9) O 

MAT 

— 

— 

— 

25 (3.5) O 

21 (3.6) O 

26(3.5) O 

21 (3.5) O 

SCI 

— 

— 

— 

26 (3.4) O 

23 (3.5) O 

26 (3.4) O 

23 (3.5) O 

REA 

— 

— 

— 

-1 (6.4) 

0 (6.3) 

-2 (6.0) 

-1 (5.9) 

MAT 

— 

— 

— 

7 (7.0) 

9 (7.0) 

6 (6.4) 

8 (6.3) 

SCI 

— 

— 

— 

-4 (6.7) 

-3 (6.7) 

-5 (6.3) 

-4 (6.3) 


REA - Reading 
MAT - Mathematics 
SCI -Science 


O Coefficient significantly greater than zero. 
® Coefficient significantly less than zero. 



Percentage of Variance Explained 


School Explanatory Models Home 

School School School Environment Background 

Environment Instruction and Instruction Control Model 


School Explanatory with Home Background Control Models 


School 

Environment 


School 

Instruction 


School Environment 
and Instruction 


Reading 

Between Schools (9%) 

25 

2 

28 

55 

58 

57 

60 

Within Schools (91%) 

— 

— 

— 

18 

18 

18 

18 

Total 

2 

0 

2 

21 

22 

21 

22 

Mathematics 

Between Schools (16%) 

22 

1 

24 

37 

42 

38 

44 

Within Schools (84%) 

— 

— 

— 

20 

20 

20 

20 

Total 

3 

0 

4 

23 

23 

23 

24 

Science 

Between Schools (13%) 

26 

2 

28 

46 

52 

48 

54 

Within Schools (87%) 

— 

— 

— 

20 

20 

20 

20 

Total 

3 

0 

4 

24 

24 

24 

25 


( ) Percentage of available variance shown in parentheses. 
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Exhibit 3.7: School Effectiveness Models for Reading, Mathematics, and Science - Azerbaijan 201ll 



HLM Regression Coefficients 


School Explanatory Models 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 

School Environment 



REA 

11 (4.5) 

o 

— 

8 (4.3) 

— 

10 (4.7) O 

— 

7 (4.6) 


Schools Are Safe and Orderly 

MAT 

16 (5.6) 

o 

— 

9 (5.5) 

— 

15(5.3) O 

— 

9 (5.1) 



SCI 

17 (5.8) 

o 

— 

11 (5.1) O 

— 

16 (5.8) O 

— 

10 (5.2) 



REA 

6 (3.6) 


— 

6(2.8) O 

— 

6 (3.7) 

— 

6 (2.8) 

o 

Schools Support Academic Success 

MAT 

5 (4.8) 


— 

6 (3.5) 

— 

6 (4.9) 

— 

5 (3.8) 



SCI 

7 (4.6) 


— 

7(3.2) O 

— 

6 (4.9) 

— 

6 (3.5) 



REA 

-1 (4.4) 


— 

-1 (3.2) 

— 

-2 (4.5) 

— 

-1 (3.5) 


Adequate Environment and Resources 

MAT 

5 (5.7) 


— 

5 (3.9) 

— 

5 (5.6) 

— 

3 (4.0) 



SCI 

0 (6.5) 


— 

0 (4.0) 

— 

0 (6.5) 

— 

0 (4.1) 


School Instruction 


REA 

— 


1 (2.8) 

1 (2.7) 

— 

— 

1 (2.8) 

1 (2.7) 


Early Emphasis on Reading Skills 

MAT 

— 


-4 (3.6) 

-3 (3.4) 

— 

— 

-4 (3.8) 

-4 (3.6) 



SCI 

— 


0 (3.3) 

0 (3.2) 

— 

— 

0 (3.2) 

0 (3.1) 


Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 


34 (9.3) O 

31 (8.9) O 

— 

— 

36 (8.3) O 

33 (7.8) 

o 

MAT 

— 


57 (10.3) O 

53 (10.0) O 

— 

— 

63 (8.4) O 

59 (8.2) 

o 

SCI 

— 


59 (9.4) O 

56 (9.0) O 

— 

— 

62 (8.3) O 

58 (8.0) 

o 


Home Background Control Variables 

Students within Schools 



REA 

— 

— 

— 

6(1.0) O 

6(1.0) O 

6 (1.0) 

o 

6 (1.0) 

o 

Home Resources for Learning 

MAT 

— 

— 

— 

9(1.2) O 

9(1.2) O 

9 (1.2) 

o 

9 (1.2) 

o 


SCI 

— 

— 

— 

9 (1.1) O 

9 (1.1) O 

9 (1.1) 

o 

9 (1.1) 

o 


REA 

— 

— 

— 

4 (0.8) O 

4 (0.8) O 

4 (0.8) 

o 

4 (0.8) 

o 

Early Literacy/Numeracy Tasks 

MAT 

— 

— 

— 

4 (0.9) O 

4 (0.9) O 

4 (0.9) 

o 

4 (0.9) 

o 


SCI 

— 

— 

— 

5 (1.0) O 

5 (1.0) O 

5 (1.0) 

o 

5 (1.0) 

o 

Between Schools 

School Average of 
Home Resources for Learning 

REA 

— 

— 

— 

13 (7.6) 

8 (7.6) 

18 (6.5) 

o 

13 (6.4) 

o 

MAT 

— 

— 

— 

12 (10.4) 

7 (9.9) 

20 (8.5) 

o 

16 (8.2) 


SCI 

— 

— 

— 

16 (10.5) 

10 (10.2) 

24 (8.6) 

o 

19 (8.1) 

o 

School Average of 

Early Literacy/Numeracy Tasks 

REA 

— 

— 

— 

-11 (7.4) 

-11 (7.5) 

-13 (7.2) 


-13 (7.3) 


MAT 

— 

— 

— 

-11 (8.8) 

-10 (8.3) 

-15 (7.8) 


-14 (7.5) 


SCI 

— 

— 

— 

-10 (8.6) 

-9 (8.1) 

-13 (7.9) 


-12 (7.6) 



() Standard errors appear in parentheses. REA -Reading O Coefficient significantly greaterthan zero. 

MAT - Mathematics ® Coefficient significantly less than zero. 

SCI -Science 



| Percentage of Variance Explained 

Source of Variance 

| School Explanatory Models ■ 

Home 

| School Explanatory with Home Background Control Models 


School 

School 

School Environment 

Background 

School 

School 

School Environment 


Environment 

Instruction 

and Instruction 

Control Model 

Environment 

Instruction 

and Instruction 


Reading 


Between Schools (44%) 

10 

20 

26 

4 

13 

25 

30 

Within Schools (56%) 

— 

— 

— 

3 

3 

3 

3 

Total 

4 

9 

11 

3 

7 

13 

15 

Mathematics 

Between Schools (48%) 

10 

29 

34 

0 

10 

31 

35 

Within Schools (52%) 

— 

— 

— 

5 

5 

5 

5 

Total 

5 

14 

17 

3 

7 

17 

19 

Science 

Between Schools (49%) 

11 

33 

39 

1 

12 

36 

41 

Within Schools (51%) 

— 

— 

— 

5 

5 

5 

5 

Total 

5 

16 

19 

3 

8 

20 

22 


( ) Percentage of available variance shown in parentheses. 
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Exhibit 3.8: School Effectiveness Models for Reading, Mathematics, and Science - 
Chinese Taipaei 


HLM Regression Coefficients 


TIMSS &PIRLSI 
201ll 



School Explanatory Models :}■ 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 

School Environment 



REA 

5 (3.8) 


— 


4 (3.5) 

— 


5 (2.2) 

O 

— 

5 (2.2) 

O 

Schools Are Safe and Orderly 

MAT 

4 (3.5) 


— 


3 (3.3) 

— 


5 (2.1) 

O 

— 

4 (2.1) 



SCI 

4 (3.5) 


— 


4 (3.3) 

— 


5 (1.9) 

O 

— 

5 (1.9) 

O 


REA 

6 (2.4) 

O 

— 


6(2.4) O 

— 


0 (1.3) 


— 

0 (1.3) 


Schools Support Academic Success 

MAT 

6 (2.3) 

O 

— 


6(2.2) O 

— 


0 (1.5) 


— 

0 (1.5) 



SCI 

6 (2.2) 

O 

— 


5(2.2) O 

— 


-1 (1.3) 


— 

-1 (1.3) 



REA 

0 (2.0) 


— 


0 (2.0) 

— 


-1 (1.4) 


— 

-1 (1.3) 


Adequate Environment and Resources 

MAT 

0 (2.2) 


— 


1 (2.1) 

— 


0 (1.6) 


— 

0 (1.5) 



SCI 

0 (2.1) 


— 


1 (2.0) 

— 


0 (1.4) 


— 

1 (1.3) 


School Instruction 


REA 

— 


-1 (1.5) 


-1 (1.4) 

— 


— 


-1 (0.8) 

-1 (0.7) 


Early Emphasis on Reading Skills 

MAT 

— 


0 (1.5) 


-1 (1.3) 

— 


— 


-1 (0.8) 

-1 (0.8) 



SCI 

— 


0 (1.5) 


-1 (1.3) 

— 


— 


-1 (0.8) 

-1 (0.8) 


Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 


12 (3.5) 

O 

10 (3.3) O 

— 


— 


4 (2.5) 

4 (2.5) 


MAT 

— 


12 (3.7) 

O 

10 (3.7) O 

— 


— 


4 (2.6) 

4 (2.6) 


SCI 

— 


11 (3.5) 

O 

8(3.5) O 

— 


— 


2 (2.6) 

2 (2.5) 


Home Background Control Variables 













Students within Schools 















REA 

— 


— 


— 

10 (0.7) 

O 

10 (0.7) 

O 

10 (0.7) O 

10 (0.7) 

O 

Home Resources for Learning 

MAT 

— 


— 


— 

12 (0.8) 

O 

12 (0.8) 

O 

12 (0.8) O 

12 (0.8) 

O 


SCI 

— 


— 


— 

12 (0.7) 

O 

12 (0.7) 

O 

12 (0.7) O 

12 (0.7) 

O 


REA 

— 


— 


— 

14 (1.0) 

O 

14 (1.0) 

O 

14 (1.0) O 

14 (1.0) 

O 

Early Literacy/Numeracy Tasks 

MAT 

— 


— 


— 

17 (1.1) 

O 

17 (1.1) 

O 

17 (1.1) O 

17 (1.1) 

O 


SCI 

— 


— 


— 

15 (1.2) 

O 

15 (1.2) 

O 

15 (1.2) O 

15 (1.2) 

O 

Between Schools 

School Average of 
Home Resources for Learning 

REA 

— 


— 


— 

20 (2.0) 

O 

20 (2.1) 

O 

20 (2.1) O 

20 (2.1) 

O 

MAT 

— 


— 


— 

22 (2.5) 

O 

22 (2.5) 

O 

22 (2.6) O 

22 (2.6) 

O 

SCI 

— 


— 


— 

22 (2.1) 

O 

22 (2.2) 

O 

22 (2.1) O 

22 (2.2) 

O 

School Average of 

Early Literacy/Numeracy Tasks 

REA 

— 


— 


— 

27 (8.7) 

O 

27 (7.5) 

O 

26 (8.4) O 

25 (7.3) 

O 

MAT 

— 


— 


— 

23 (6.3) 

O 

22 (6.6) 

O 

21 (6.0) O 

20 (6.4) 

O 

SCI 

— 


— 


— 

21 (6.6) 

O 

21 (6.0) 

O 

20 (6.3) O 

20 (5.8) 

O 

( ) Standard errors appear in parentheses. 

REA - 

Reading 

O Coefficient significantly greater than zero. 









MAT- 

Mathematics 

® Coefficient significantly less than zero. 









SCI - 

Science 














Percentage of Variance Explained 


School Explanatory Models H ome School Explanatory with Home Background Control Models 

School School School Environment Background School School School Environment 

Environment Instruction and Instruction Control Model Environment Instruction and Instruction 


Reading 

Between Schools (10%) 

14 

11 

21 

75 

77 

77 

78 

Within Schools (90%) 

— 

— 

— 

15 

15 

15 

15 

Total 

2 

1 

2 

21 

21 

21 

21 

Mathematics 

Between Schools (11%) 

13 

10 

20 

74 

76 

76 

78 

Within Schools (89%) 

— 

— 

— 

20 

20 

20 

20 

Total 

1 

1 

2 

25 

26 

26 

26 

Science 

Between Schools (10%) 

13 

8 

18 

79 

81 

80 

82 

Within Schools (90%) 

— 

— 

— 

20 

20 

20 

20 

Total 

1 

1 

2 

26 

26 

26 

26 


( ) Percentage of available variance shown in parentheses. 
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Exhibit 3.9: School Effectiveness Models for Reading, Mathematics, and Science - Croatia 2011 



HLM Regression Coefficients 


School Explanatory Models 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 

School Environment 



REA 

2 (2.8) 

— 

3 (2.7) 


— 


4 (1.9) 


— 


4 (1.9) 

o 

Schools Are Safe and Orderly 

MAT 

1 (3.2) 

— 

2 (3.0) 


— 


3 (2.0) 


— 


3 (2.0) 



SCI 

2 (2.9) 

— 

3 (2.7) 


— 


4 (2.1) 


— 


4 (2.0) 



REA 

5 (1.9) O 

— 

5 (1.9) 

o 

— 


0 (1.2) 


— 


0 (1.2) 


Schools Support Academic Success 

MAT 

5 (2.1) O 

— 

4 (2.0) 

o 

— 


-1 (1.3) 


— 


-1 (1.2) 



SCI 

4(1.7) O 

— 

4 (1.7) 

o 

— 


-1 (1.D 


— 


-1 (1.1) 



REA 

-3 (1.5) 

— 

-2 (1.6) 


— 


-1 (1.2) 


— 


-1 (1.2) 


Adequate Environment and Resources 

MAT 

-1 (1.7) 

— 

0 (1.8) 


— 


1 (1.2) 


— 


1 (1.3) 



SCI 

-3 (1.5) 

— 

-2 (1.6) 


— 


-1 (1.2) 


— 


-1 (1.2) 


School Instruction 


REA 

— 

1 (1.5) 

0 (1.6) 


— 


— 


0 (1.0) 


0 (1.1) 


Early Emphasis on Reading Skills 

MAT 

— 

2 (1.8) 

1 (1.8) 


— 


— 


0 (1.2) 


0 (1.2) 



SCI 

— 

1 (1.4) 

0 (1.4) 


— 


— 


-1 (1.0) 


-1 (1.0) 


Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 

-7(3.5) ® 

-8 (3.6) 

® 

— 


— 


-2 (2.7) 


-2 (2.6) 


MAT 

— 

-8 (4.3) 

-9 (4.6) 

® 

— 


— 


-1 (2.6) 


-2 (2.7) 


SCI 

— 

-7(3.4) ® 

-8 (3.6) 

® 

— 


— 


-2 (2.7) 


-2 (2.7) 


Home Background Control Variables 

Students within Schools 


REA 

— 

— 

— 


12 (0.8) 

o 

12 (0.8) 

o 

12 (0.8) 

o 

12 (0.8) 

o 

Home Resources for Learning 

MAT 

— 

— 

— 


11 (0.8) 

o 

11 (0.8) 

o 

11 (0.8) 

o 

11 (0.8) 

o 


SCI 

— 

— 

— 


12 (1.0) 

o 

12 (1.0) 

o 

12 (1.0) 

o 

12 (1.0) 

o 


REA 

— 

— 

— 


12 (0.7) 

o 

12 (0.7) 

o 

12 (0.7) 

o 

12 (0.7) 

o 

Early Literacy/Numeracy Tasks 

MAT 

— 

— 

— 


16 (0.9) 

o 

16 (0.9) 

o 

16 (0.9) 

o 

16 (0.9) 

o 


SCI 

— 

— 

— 


10 (0.9) 

o 

10 (0.9) 

o 

10 (0.9) 

o 

10 (0.9) 

o 

Between Schools 

School Average of 
Home Resources for Learning 

REA 

— 

— 

— 


23 (2.1) 

o 

22 (2.0) 

o 

23 (2.0) 

o 

22 (1.9) 

o 

MAT 

— 

— 

— 


24 (2.4) 

o 

25 (2.6) 

o 

24 (2.4) 

o 

25 (2.6) 

o 

SCI 

— 

— 

— 


21 (2.7) 

o 

21 (2.6) 

o 

21 (2.7) 

o 

21 (2.7) 

o 

School Average of 

Early Literacy/Numeracy Tasks 

REA 

— 

— 

— 


9 (4.3) 

o 

10 (4.2) 

o 

9 (4.3) 

o 

10 (4.2) 

o 

MAT 

— 

— 

— 


15 (5.5) 

o 

15 (5.3) 

o 

15 (5.5) 

o 

15 (5.2) 

o 


() Standard errors appear in parentheses. REA -Reading O Coefficient significantlygreaterthan zero. 

MAT - Mathematics ® Coefficient significantly less than zero. 

SCI -Science 



ij Percentage of Variance Explained 

Source of Variance 

|j School Explanatory Models |j 

Home 

| School Explanatory with Home Background Control Models 


School 

School 

School Environment 

Background 

School 

School 

School Environment 


Environment 

Instruction 

and Instruction 

Control Model 

Environment 

Instruction 

and Instruction 


Reading 


Between Schools (10%) 

11 

6 

16 

73 

74 

73 

74 

Within Schools (90%) 

— 

— 

— 

16 

16 

16 

16 

Total 

1 

1 

2 

22 

22 

22 

22 

Mathematics 

Between Schools (13%) 

6 

6 

12 

69 

70 

70 

70 

Within Schools (87%) 

— 

— 

— 

21 

21 

21 

21 

Total 

1 

1 

2 

27 

27 

27 

27 

Science 

Between Schools (11%) 

9 

5 

13 

62 

64 

62 

64 

Within Schools (89%) 

— 

— 

— 

16 

16 

16 

16 

Total 

1 

1 

1 

21 

21 

21 

21 


( ) Percentage of available variance shown in parentheses. 
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Exhibit 3.10: School Effectiveness Models for Reading, Mathematics, and Science - 
Czech Republic 


HLM Regression Coefficients 


TIMSS &PIRLSI 
201ll 



School Explanatory Models :}■ 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 


School Environment 



REA 

2 (5.4) 

— 

2 (5.5) 

— 

4 (3.0) 

— 

4 (2.9) 

Schools Are Safe and Orderly 

MAT 

1 (7.7) 

— 

2 (7.8) 

— 

5 (3.9) 

— 

5 (3.8) 


SCI 

3 (5.9) 

— 

3 (5.9) 

— 

5 (3.2) 

— 

5 (3.2) 


REA 

2 (2.4) 

— 

3 (2.4) 

— 

-1 (1.5) 

— 

-2 (1.5) 

Schools Support Academic Success 

MAT 

3 (3.0) 

— 

3 (2.9) 

— 

-2 (1.5) 

— 

-2 (1.6) 


SCI 

3 (2.5) 

— 

3 (2.5) 

— 

-1 (1.5) 

— 

-2 (1.6) 


REA 

-4 (2.9) 

— 

-5 (3.0) 

— 

-2 (1.6) 

— 

-1 (1.5) 

Adequate Environment and Resources 

MAT 

-5 (3.8) 

— 

-5 (3.8) 

— 

-2 (2.0) 

— 

-2 (1.9) 


SCI 

-4 (3.2) 

— 

-4 (3.2) 

— 

-1 (1.7) 

— 

-1 (1.7) 

School Instruction 


REA 

— 

0 (1.4) 

-1 (1.3) 

— 

— 

1 (0.9) 

1 (0.9) 

Early Emphasis on Reading Skills 

MAT 

— 

0 (1.5) 

-1 (1.5) 

— 

— 

1 (0.9) 

1 (0.9) 


SCI 

— 

0 (1.4) 

-1 (1.3) 

— 

— 

1 (0.9) 

1 (0.8) 

Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 

-1 (4.7) 

-2 (4.5) 

— 

— 

1 (3.3) 

1 (3.2) 

MAT 

— 

-1 (5.5) 

-2 (5.2) 

— 

— 

1 (3.4) 

1 (3.4) 

SCI 

— 

1 (4.2) 

0 (4.2) 

— 

— 

3 (3.1) 

3 (3.2) 


Home Background Control Variables 

Students within Schools 



REA 

— 

— 

— 

15 (1.1) 

O 

15 (1.1) 

O 

15 (1.1) 

O 

15 (1.1) 

O 

Home Resources for Learning 

MAT 

— 

— 

— 

17 (1.2) 

O 

17 (1.2) 

O 

17 (1.2) 

O 

17 (1.2) 

O 


SCI 

— 

— 

— 

17 (1.3) 

O 

17 (1.3) 

O 

17 (1.3) 

O 

17 (1.3) 

O 


REA 

— 

— 

— 

7 (0.8) 

O 

7 (0.8) 

O 

7 (0.8) 

O 

7 (0.8) 

O 

Early Literacy/Numeracy Tasks 

MAT 

— 

— 

— 

9 (0.9) 

O 

9 (0.9) 

O 

9 (0.9) 

O 

9 (0.9) 

O 


SCI 

— 

— 

— 

7 (1.0) 

O 

7 (1.0) 

O 

7 (1.0) 

O 

7 (1.0) 

O 


Between Schools 


School Average of 
Home Resources for Learning 

REA 

— 

— 

— 

31 (4.5) 

O 

31 (4.3) 

O 

31 (4.5) 

O 

31 (4.4) 

O 

MAT 

— 

— 

— 

37 (5.9) 

O 

38 (5.6) 

O 

38 (6.0) 

O 

38 (5.8) 

O 

SCI 

— 

— 

— 

31 (4.9) 

O 

32 (4.8) 

O 

32 (4.9) 

O 

32 (4.9) 

O 

School Average of 

Early Literacy/Numeracy Tasks 

REA 

— 

— 

— 

16 (5.5) 

O 

17 (5.2) 

O 

16 (5.7) 

O 

16 (5.4) 

O 

MAT 

— 

— 

— 

26 (7.7) 

O 

26 (7.2) 

O 

25 (7.9) 

O 

26 (7.5) 

O 

SCI 

— 

— 

— 

19 (5.7) 

O 

19 (5.4) 

O 

18 (6.0) 

O 

19 (5.6) 

O 


() Standard errors appear in parentheses. REA -Reading O Coefficient significantly greater than zero. 

MAT - Mathematics ® Coefficient significantly less than zero. 

SCI -Science 


Source of Variance 

1 Percentage of Variance Explained 

j School Explanatory Models f 

Home 

| School Explanatory with Home Background Control Models 


School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


Reading 


Between Schools (15%) 

6 

0 

7 

76 

77 

76 

77 

Within Schools (85%) 

— 

— 

— 

16 

16 

16 

16 

Total 

1 

0 

1 

25 

25 

25 

25 

Mathematics 

Between Schools (21%) 

5 

0 

6 

77 

78 

77 

79 

Within Schools (79%) 

— 

— 

— 

19 

19 

19 

19 

Total 

1 

0 

1 

31 

31 

31 

31 

Science 

Between Schools (14%) 

6 

0 

6 

73 

75 

74 

76 

Within Schools (86%) 

— 

— 

— 

17 

17 

17 

17 

Total 

1 

0 

1 

25 

25 

25 

25 


( ) Percentage of available variance shown in parentheses. 
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Exhibit 3.11: School Effectiveness Models for Reading, Mathematics, and Science - Finland ! IMSS I 1 



HLM Regression Coefficients 


School Explanatory Models 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 

School Environment 



REA 

9 (3.2) 

o 

— 

9(3.2) O 

— 


10 (2.6) 

o 

— 


10 (2.6) 

o 

Schools Are Safe and Orderly 

MAT 

9 (3.1) 

o 

— 

9(3.1) O 

— 


10 (2.6) 

o 

— 


10 (2.5) 

o 


SCI 

11 (2.6) 

o 

— 

11 (2.6) O 

— 


10 (2.3) 

o 

— 


11 (2.3) 

o 


REA 

5 (2.4) 

o 

— 

5 (2.5) 

— 


0 (2.2) 


— 


-1 (2.3) 


Schools Support Academic Success 

MAT 

5 (2.2) 

o 

— 

5(2.3) O 

— 


0 (2.2) 


— 


-1 (2.1) 



SCI 

6 (2.2) 

o 

— 

6(2.3) O 

— 


2 (2.1) 


— 


2 (2.2) 



REA 

-2 (1.8) 


— 

-2 (1.8) 

— 


1 (1.6) 


— 


1 (1.5) 


Adequate Environment and Resources 

MAT 

-1 (1.6) 


— 

0 (1.6) 

— 


2 (1.5) 


— 


3 (1.5) 



SCI 

-3 (1.6) 


— 

-3 (1.5) 

— 


0 (1.5) 


— 


0 (1.5) 


School Instruction 


REA 

— 


1 (1.9) 

0 (1.7) 

— 


— 


1 (1.7) 


1 (1.5) 


Early Emphasis on Reading Skills 

MAT 

— 


2 (2.5) 

1 (2.3) 

— 


— 


2 (2.4) 


2 (2.1) 



SCI 

— 


1 (2.0) 

0 (1.9) 

— 


— 


1 (1.7) 


1 (1.6) 


Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 


-4 (5.4) 

-3 (5.2) 

— 


— 


1 (4.6) 


0 (4.2) 


MAT 

— 


-3 (6.2) 

-3 (6.2) 

— 


— 


-1 (5.7) 


-1 (5.4) 


SCI 

— 


-6 (5.3) 

-5 (5.0) 

— 


— 


0 (4.4) 


-1 (4.0) 


Home Background Control Variables 

Students within Schools 


REA 

— 


— 

— 

11 (1.1) 

o 

11 (1.1) 

o 

11 (1.0) 

o 

11 (1.0) 

o 

Home Resources for Learning 

MAT 

— 


— 

— 

10 (0.9) 

o 

10 (0.9) 

o 

10 (0.9) 

o 

10 (0.9) 

o 


SCI 

— 


— 

— 

11 (1.0) 

o 

11 (1.0) 

o 

11 (1.0) 

o 

11 (1.0) 

o 


REA 

— 


— 

— 

15 (1.0) 

o 

15 (1.0) 

o 

15 (1.0) 

o 

15 (1.0) 

o 

Early Literacy/Numeracy Tasks 

MAT 

— 


— 

— 

18 (0.8) 

o 

18 (0.8) 

o 

18 (0.8) 

o 

18 (0.8) 

o 


SCI 

— 


— 

— 

12 (1.0) 

o 

12 (1.0) 

o 

12 (1.0) 

o 

12 (1.0) 

o 

Between Schools 

School Average of 
Home Resources for Learning 

REA 

— 


— 

— 

19 (4.1) 

o 

19 (4.3) 

o 

19 (3.9) 

o 

19 (4.0) 

o 

MAT 

— 


— 

— 

15 (5.3) 

o 

15 (5.8) 

o 

15 (4.9) 

o 

16 (5.1) 

o 

SCI 

— 


— 

— 

20 (4.7) 

o 

18 (4.4) 

o 

20 (4.9) 

o 

18 (4.3) 

o 

School Average of 

Early Literacy/Numeracy Tasks 

REA 

— 


— 

— 

10 (6.2) 


10 (5.4) 


10 (6.3) 


10 (5.5) 


MAT 

— 


— 

— 

15 (6.1) 

o 

15 (5.6) 

o 

15 (6.2) 

o 

15 (5.5) 

o 

SCI 

— 


— 

— 

7 (5.9) 


6 (5.3) 


7 (6.1) 


6 (5.3) 



() Standard errors appear in parentheses. REA -Reading O Coefficient significantlygreaterthan zero. 


MAT 

SCI 

-Mathematics ® Coefficient significantly less than zero. 
- Science 







Percentage of Variance Explained 

Source of Variance 


School Explanatory Models 


Home 


School Explanatory with Home Background Control Models 



School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


Background 
Control Model 


School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Reading 








Between Schools (7%) 

39 

3 

40 

30 

58 

31 

58 

Within Schools (93%) 

— 

— 

— 

22 

22 

22 

22 

Total 

3 

0 

3 

23 

25 

23 

25 

Mathematics 








Between Schools (9%) 

32 

4 

34 

18 

42 

20 

44 

Within Schools (91%) 

— 

— 

— 

28 

28 

28 

28 

Total 

3 

0 

3 

27 

29 

27 

29 

Science 








Between Schools (10%) 

40 

4 

42 

25 

S3 

26 

54 

Within Schools (90%) 

— 

— 

— 

20 

20 

20 

20 

Total 

4 

0 

4 

20 

23 

20 

23 


( ) Percentage of available variance shown in parentheses. 
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Exhibit 3.12: School Effectiveness Models for Reading, Mathematics, and Science - Georgia 


TIMSS & PIRLS 
2011 


4 th 

Grade 



| HLM Regression Coefficients 

Variables 

| School Explanatory Models ) 

Home 

| School Explanatory with Home Background Control Models 


School 

School 

School Environment 

Background 

School 

School 

School Environment 


Environment 

Instruction 

and Instruction 

Control Model 

Environment 

Instruction 

and Instruction 


School Explanatory Variables 


School Environment 



REA 

5 (4.2) 

— 

3 (3.7) 

— 

3 (3.5) 

— 

2 (3.2) 

Schools Are Safe and Orderly 

MAT 

0 (5.8) 

— 

-2 (4.8) 

— 

-1 (5.4) 

— 

-2 (4.5) 


SCI 

0 (5.1) 

— 

-2 (4.3) 

— 

-2 (4.6) 

— 

-4 (4.0) 


REA 

9(3.8) O 

— 

7 (3.6) 

— 

2 (3.3) 

— 

1 (3.0) 

Schools Support Academic Success 

MAT 

16 (5.9) O 

— 

13 (5.5) O 

— 

12 (6.8) 

— 

10 (6.2) 


SCI 

12 (4.8) O 

— 

10 (4.4) O 

— 

7 (5.7) 

— 

6 (5.2) 


REA 

-4 (3.8) 

— 

-6 (3.7) 

— 

-7(3.2) ® 

— 

-7(3.1) ® 

Adequate Environment and Resources 

MAT 

-8 (4.9) 

— 

-9 (4.9) 

— 

-11 (4.6) ® 

— 

-11 (4.5) ® 


SCI 

-7 (4.6) 

— 

-8 (4.6) 

— 

-9 (4.1) ® 

— 

-10 (4.0) ® 

School Instruction 


REA 

— 

1 (3.1) 

0 (3.3) 

— 

— 

-1 (2.6) 

0 (2.7) 

Early Emphasis on Reading Skills 

MAT 

— 

-2 (4.1) 

-2 (4.1) 

— 

— 

-3 (3.4) 

-2 (3.4) 


SCI 

— 

0 (3.5) 

0 (3.6) 

— 

— 

-2 (3.1) 

-1 (3.2) 

Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 

31 (9.2) O 

29 (9.5) O 

— 

— 

23 (8.8) O 

24 (8.7) O 

MAT 

— 

40 (12.5) O 

37 (12.9) O 

— 

— 

35 (11.8) O 

35 (11.4) O 

SCI 

— 

35 (11.6) O 

34 (12.0) O 

— 

— 

29 (11.2) O 

31 (10.8) O 


Home Background Control Variables 

Students within Schools 



REA 

— 

— 

— 

11 (1.2) 

o 

11 (1.2) 

o 

11 (1.2) 

o 

11 (1.2) 

o 

Home Resources for Learning 

MAT 

— 

— 

— 

10 (1.5) 

o 

10 (1.5) 

o 

10 (1.5) 

o 

10 (1.5) 

o 


SCI 

— 

— 

— 

11 (1.3) 

o 

11 (1.3) 

o 

11 (1.3) 

o 

11 (1.3) 

o 


REA 

— 

— 

— 

9 (0.8) 

o 

9 (0.8) 

o 

9 (0.8) 

o 

9 (0.8) 

o 

Early Literacy/Numeracy Tasks 

MAT 

— 

— 

— 

8 (1.1) 

o 

8 (1.0) 

o 

8 (1.0) 

o 

8 (1.0) 

o 


SCI 

— 

— 

— 

9 (1.0) 

o 

9 (1.1) 

o 

9 (1.0) 

o 

9 (1.1) 

o 


Between Schools 


School Average of 
Home Resources for Learning 

REA 

— 

— 

— 

20(3.2) O 

21 (3.4) O 

18 (3.2) O 

20 (3.4) 

o 

MAT 

— 

— 

— 

21 (4.7) O 

19(5.1) O 

18 (4.6) O 

18 (5.1) 

o 

SCI 

— 

— 

— 

17 (4.4) O 

17 (4.9) O 

15 (4.4) O 

15 (4.9) 

o 

School Average of 

Early Literacy/Numeracy Tasks 

REA 

— 

— 

— 

5 (5.4) 

4 (5.3) 

4 (5.0) 

4 (4.9) 


MAT 

— 

— 

— 

-1 (9.2) 

-2 (9.1) 

-2 (8.2) 

-3 (8.3) 


SCI 

— 

— 

— 

4 (7.5) 

3 (7.5) 

3 (6.7) 

3 (6.8) 



() Standard errors appear in parentheses. REA -Reading O Coefficient significantly greater than zero. 

MAT - Mathematics ® Coefficient significantly less than zero. 

SCI -Science 



|J Percentage of Variance Explained 

Source of Variance 

| School Explanatory Models | 

Home 

| School Explanatory with Home Background Control Models 


School 

School 

School Environment 

Background 

; School 

School 

School Environment 


Environment 

Instruction 

and Instruction 

Control Model 

Environment 

Instruction 

and Instruction 


Reading 


Between Schools (25%) 

9 

16 

22 

34 

37 

41 

45 

Within Schools (75%) 

— 

— 

— 

11 

11 

11 

11 

Total 

2 

4 

5 

17 

18 

19 

20 

Mathematics 

Between Schools (38%) 

10 

14 

21 

11 

16 

20 

25 

Within Schools (62%) 

— 

— 

— 

11 

11 

11 

11 

Total 

4 

5 

8 

11 

13 

14 

16 

Science 

Between Schools (32%) 

8 

15 

21 

16 

21 

25 

30 

Within Schools (68%) 

— 

— 

— 

12 

12 

12 

12 

Total 

3 

5 

7 

13 

15 

16 

18 


( ) Percentage of available variance shown in parentheses. 
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TIMSS ' y^jy^ y ' ^1 

Exhibit 3.13: School Effectiveness Models for Reading, Mathematics, and Science - Germany A 201 1! 



HLM Regression Coefficients 


School Explanatory Models 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 

School Environment 



REA 

9(3.3) © 

— 


9(3.3) O 

— 

9 (3.1) 

o 

— 

9 (3.1) 

o 

Schools Are Safe and Orderly 

MAT 

6 (3.1) 

— 


7 (3.1) O 

— 

6 (3.0) 

o 

— 

7 (2.9) 

o 


SCI 

8(3.4) O 

— 


9(3.5) O 

— 

8 (3.2) 

o 

— 

8 (3.2) 

o 


REA 

16 (3.4) O 

— 


15 (3.0) O 

— 

6 (3.2) 


— 

6 (2.9) 


Schools Support Academic Success 

MAT 

15 (3.2) O 

— 


14(2.7) O 

— 

7 (3.0) 

o 

— 

6 (2.8) 

o 


SCI 

17 (3.2) O 

— 


15 (2.9) O 

— 

7 (3.0) 

o 

— 

7 (2.8) 

o 


REA 

-1 (2.1) 

— 


-1 (2.0) 

— 

1 (1.9) 


— 

1 (1.9) 


Adequate Environment and Resources 

MAT 

-1 (1.9) 

— 


-2 (1.8) 

— 

0 (1.7) 


— 

0 (1.7) 



SCI 

-1 (1.9) 

— 


-1 (1.8) 

— 

1 (1.7) 


— 

1 (1.7) 


School Instruction 


REA 

— 

8 (2.8) 

o 

5(2.3) O 

— 

— 


5 (2.1) o 

4 (1.9) 

o 

Early Emphasis on Reading Skills 

MAT 

— 

7 (2.7) 

o 

4(2.2) O 

— 

— 


4 (2.1) 

3 (1.9) 



SCI 

— 

7 (2.8) 

o 

4 (2.3) 

— 

— 


4(2.1) o 

3 (2.0) 


Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 

-4 (4.8) 


-3 (3.7) 

— 

— 


1 (4.3) 

-1 (3.8) 


MAT 

— 

-6 (4.5) 


-5 (3.6) 

— 

— 


-2 (4.2) 

-4 (3.6) 


SCI 

— 

-6 (4.9) 


-5 (3.8) 

— 

— 


-1 (4.3) 

-3 (3.8) 


Home Background Control Variables 











Students within Schools 













REA 

— 

— 


— 

16 (0.8) O 

16 (0.8) 

o 

16 (0.8) O 

16 (0.8) 

o 

Home Resources for Learning 

MAT 

— 

— 


— 

13 (0.8) O 

13 (0.8) 

o 

13 (0.8) O 

13 (0.8) 

o 


SCI 

— 

— 


— 

17 (0.8) O 

16 (0.8) 

o 

17 (0.8) O 

16 (0.8) 

o 


REA 

— 

— 


— 

7(0.9) O 

7 (0.9) 

o 

7(0.9) O 

7 (0.9) 

o 

Early Literacy/Numeracy Tasks 

MAT 

— 

— 


— 

9 (1.0) O 

9 (1.0) 

o 

9 (1.0) O 

9 (1.0) 

o 


SCI 

— 

— 


— 

5 (1.1) O 

5 (1.1) 

o 

5 (1.1) O 

5 (1.1) 

o 

Between Schools 

School Average of 
Home Resources for Learning 

REA 

— 

— 


— 

35 (3.9) O 

26 (3.7) 

o 

33 (3.6) O 

25 (3.8) 

o 

MAT 

— 

— 


— 

31 (3.9) O 

22 (3.8) 

o 

30 (3.8) O 

21 (3.9) 

o 

SCI 

— 

— 


— 

36 (4.1) O 

26 (3.9) 

o 

34 (3.9) O 

25 (4.0) 

o 

School Average of 

Early Literacy/Numeracy Tasks 

REA 

— 

— 


— 

8 (10.4) 

11 (9.4) 


6 (10.0) 

10 (9.0) 


MAT 

— 

— 


— 

10 (9.5) 

13 (8.5) 


9 (9.3) 

13 (8.2) 


SCI 

— 

— 


— 

5 (10.3) 

9 (9.2) 


4 (10.0) 

8 (8.9) 



() Standard errors appear in parentheses. REA -Reading © Coefficient significantlygreaterthan zero. 


MAT 

SCI 

- Mathematics ® Coefficient significantly less than zero. 

- Science 







Percentage of Variance Explained 

Source of Variance 


School Explanatory Models 


Home 


School Explanatory with Home Background Control Models 



School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


Background 
Control Model 


School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Reading 








Between Schools (24%) 

40 

10 

44 

49 

58 

53 

61 

Within Schools (76%) 

— 

— 

— 

15 

15 

15 

15 

Total 

10 

2 

11 

23 

25 

24 

26 

Mathematics 








Between Schools (27%) 

37 

9 

41 

44 

52 

47 

55 

Within Schools (73%) 

— 

— 

— 

17 

17 

17 

17 

Total 

10 

3 

11 

24 

26 

25 

27 

Science 








Between Schools (26%) 

41 

9 

44 

48 

58 

51 

59 

Within Schools (74%) 

— 

— 

— 

17 

17 

17 

17 

Total 

11 

2 

11 

25 

27 

25 

28 


( ) Percentage of available variance shown in parentheses. 
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SOURCE: lEA's Trends in International Mathematics and Science Study and Progress in International Reading Literacy Study - TIMSS and PIRLS 201 1 



























Exhibit 3.14: School Effectiveness Models for Reading, Mathematics, and Science - 
Hong Kong SAR 


HLM Regression Coefficients 


TIMSS &PIRLSI 
201ll 



School Explanatory Models | 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 


School Environment 



REA 

7 (4.0) 

— 


2 (3.9) 


— 

4(2.2) O 

— 


3 (2.1) 


Schools Are Safe and Orderly 

MAT 

6 (3.9) 

— 


1 (3.9) 


— 

3 (2.1) 

— 


2 (2.2) 



SCI 

5 (4.2) 

— 


0 (4.3) 


— 

2 (2.2) 

— 


1 (2.4) 



REA 

0 (2.6) 

— 


-4 (2.5) 


— 

-4 (1.9) 

— 


-5 (2.0) 


Schools Support Academic Success 

MAT 

3 (2.6) 

— 


-1 (2.5) 


— 

-2 (1.7) 

— 


-2 (1.7) 



SCI 

2 (2.7) 

— 


-2 (2.6) 


— 

-2 (1.8) 

— 


-3 (1.9) 



REA 

-2 (3.8) 

— 


-2 (3.5) 


— 

-1 (1.9) 

— 


-1 (1.8) 


Adequate Environment and Resources 

MAT 

-2 (3.6) 

— 


-2 (3.3) 


— 

-2 (2.5) 

— 


-2 (2.4) 



SCI 

-2 (4.2) 

— 


-2 (3.9) 


— 

-2 (2.2) 

— 


-2 (2.2) 


School Instruction 


REA 

— 

3 (1.6) 

O 

4 (1.6) 

O 

— 

— 

1 (1.1) 


1 (1.0) 


Early Emphasis on Reading Skills 

MAT 

— 

3 (1.5) 

O 

3 (1.5) 


— 

— 

0 (0.9) 


0 (0.9) 



SCI 

— 

3 (1.6) 

O 

3 (1.6) 

O 

— 

— 

1 (1.0) 


1 (TO) 


Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 

22 (4.3) 

O 

23 (5.2) 

O 

— 

— 

8 (3.3) 

O 

9 (3.5) 

O 

MAT 

— 

22 (4.4) 

O 

22 (5.0) 

O 

— 

— 

7 (3.3) 

O 

7 (3.5) 


SCI 

— 

20 (4.4) 

O 

21 (5.3) 

O 

— 

— 

4 (3.4) 


6 (3.7) 



Home Background Control Variables 

Students within Schools 



REA 

— 

— 

— 

2 (0.7) 

O 

2 (0.7) 

O 

3 (0.7) 

O 

2 (0.7) 

O 

Home Resources for Learning 

MAT 

— 

— 

— 

3 (0.7) 

O 

3 (0.7) 

O 

3 (0.7) 

O 

3 (0.7) 

O 


SCI 

— 

— 

— 

5 (0.8) 

O 

5 (0.8) 

O 

5 (0.8) 

O 

5 (0.8) 

O 


REA 

— 

— 

— 

13 (1.1) 

O 

13 (1.1) 

O 

13 (1.1) 

O 

13 (1.1) 

O 

Early Literacy/Numeracy Tasks 

MAT 

— 

— 

— 

12 (1.1) 

O 

12 (1.1) 

O 

12 (1.1) 

O 

12 (1.1) 

O 


SCI 

— 

— 

— 

15 (1.4) 

O 

15 (1.4) 

O 

15 (1.4) 

O 

15 (1.4) 

O 


Between Schools 


School Average of 
Home Resources for Learning 

REA 

— 

— 

— 

3 (2.0) 


5 (2.2) 

O 

2 (2.1) 


4 (2.2) 

O 

MAT 

— 

— 

— 

6 (2.0) 

O 

7 (2.2) 

O 

5 (2.0) 

O 

6 (2.2) 

O 

SCI 

— 

— 

— 

5 (1.9) 

O 

6 (2.1) 

O 

4 (2.0) 

O 

6 (2.1) 

O 

School Average of 

Early Literacy/Numeracy Tasks 

REA 

— 

— 

— 

62 (4.7) 

O 

60 (4.5) 

O 

59 (4.5) 

O 

56 (4.4) 

O 

MAT 

— 

— 

— 

55 (4.9) 

O 

54 (4.8) 

O 

53 (5.0) 

O 

52 (5.0) 

O 

SCI 

— 

— 

— 

60 (5.2) 

O 

58 (5.2) 

O 

58 (5.2) 

O 

56 (5.2) 

O 


() Standard errors appear in parentheses. REA -Reading O Coefficient significantly greater than zero. 

MAT - Mathematics ® Coefficient significantly less than zero. 

SCI -Science 



j Percentage of Variance Explained 

Source of Variance 

| School Explanatory Models ( 

Home 

| School Explanatory with Home Background Control Models 


School 

School 

School Environment 

Background 

School 

School 

School Environment 


Environment 

Instruction 

and Instruction 

Control Model 

Environment 

Instruction 

and Instruction 


Reading 


Between Schools (21%) 

4 

20 

22 

68 

70 

70 

73 

Within Schools (79%) 

— 

— 

— 

8 

8 

8 

8 

Total 

1 

4 

4 

20 

21 

21 

21 

Mathematics 

Between Schools (22%) 

4 

20 

20 

68 

69 

70 

71 

Within Schools (78%) 

— 

— 

— 

10 

10 

10 

10 

Total 

1 

4 

5 

23 

23 

23 

24 

Science 

Between Schools (20%) 

2 

17 

18 

69 

69 

69 

71 

Within Schools (80%) 

— 

— 

— 

11 

11 

11 

11 

Total 

0 

3 

4 

22 

23 

23 

23 


( ) Percentage of available variance shown in parentheses. 
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Exhibit 3.15: School Effectiveness Models for Reading, Mathematics, and Science - Hungary J 201 1! 



HLM Regression Coefficients 


School Explanatory Models ; 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 


School Environment 



REA 

11 (6.1) 


— 

10 (6.3) 


— 

1 (3.7) 

— 


0 (3.8) 


Schools Are Safe and Orderly 

MAT 

16 (5.5) 

o 

— 

16 (5.6) 

o 

— 

4 (3.6) 

— 


2 (3.7) 



SCI 

13 (6.3) 

o 

— 

13 (6.4) 


— 

3 (3.9) 

— 


1 (4.0) 



REA 

16 (3.9) 

o 

— 

16 (3.8) 

o 

— 

2 (2.6) 

— 


1 (2.3) 


Schools Support Academic Success 

MAT 

18 (4.0) 

o 

— 

18 (3.9) 

o 

— 

3 (2.5) 

— 


3 (2.3) 



SCI 

17 (4.2) 

o 

— 

18 (4.0) 

o 

— 

3 (2.9) 

— 


2 (2.7) 



REA 

-2 (2.2) 


— 

-3 (2.1) 


— 

1 (1.5) 

— 


1 (1.4) 


Adequate Environment and Resources 

MAT 

-3 (2.3) 


— 

-3 (2.2) 


— 

1 (1.3) 

— 


1 (1.3) 



SCI 

-3 (2.3) 


— 

-4 (2.2) 


— 

0 (1.7) 

— 


0 (1.6) 


School Instruction 


REA 

— 


0 (3.5) 

-2 (2.7) 


— 

— 

0 (1.8) 


0 (1.8) 


Early Emphasis on Reading Skills 

MAT 

— 


0 (3.6) 

-3 (2.4) 


— 

— 

0 (1.7) 


-1 (1.7) 



SCI 

— 


0 (3.7) 

-3 (2.7) 


— 

— 

0 (1.9) 


0 (2.0) 


Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 


5 (11.6) 

2 (9.3) 


— 

— 

15 (6.1) 

o 

15 (5.9) 

o 

MAT 

— 


7 (11.7) 

2 (9.3) 


— 

— 

17 (6.6) 

o 

16 (6.3) 

o 

SCI 

— 


6 (12.6) 

3 (10.3) 


— 

— 

17 (7.9) 

o 

17 (7.8) 

o 


Home Background Control Variables 

Students within Schools 



REA 

— 

— 

— 

16 (0.8) 

o 

16 (0.8) 

o 

16 (0.9) 

o 

16 (0.8) 

o 

Home Resources for Learning 

MAT 

— 

— 

— 

17 (0.8) 

o 

17 (0.8) 

o 

17 (0.8) 

o 

17 (0.8) 

o 


SCI 

— 

— 

— 

17 (0.9) 

o 

17 (0.9) 

o 

17 (0.9) 

o 

17 (0.8) 

o 


REA 

— 

— 

— 

8 (0.8) 

o 

8 (0.7) 

o 

8 (0.7) 

o 

8 (0.7) 

o 

Early Literacy/Numeracy Tasks 

MAT 

— 

— 

— 

9 (0.8) 

o 

9 (0.8) 

o 

9 (0.8) 

o 

9 (0.8) 

o 


SCI 

— 

— 

— 

6 (1.0) 

o 

6 (1.0) 

o 

6 (1.0) 

o 

6 (1.0) 

o 


Between Schools 


School Average of 
Home Resources for Learning 

REA 

— 

— 

— 

29 (2.0) O 

27(2.7) O 

29 (2.0) O 

29 (2.4) 

o 

MAT 

— 

— 

— 

35(2.1) O 

31 (2.6) O 

35(2.1) O 

32 (2.3) 

o 

SCI 

— 

— 

— 

31 (2.1) O 

28 (2.9) O 

31 (2.1) O 

30 (2.7) 

o 

School Average of 

Early Literacy/Numeracy Tasks 

REA 

— 

— 

— 

2 (7.0) 

3 (7.1) 

6 (7.6) 

6 (7.5) 


MAT 

— 

— 

— 

-3 (7.8) 

-2 (8.0) 

2 (8.6) 

2 (8.6) 


SCI 

— 

— 

— 

-1 (8.2) 

0 (8.6) 

4 (9.0) 

4 (9.1) 



() Standard errors appear in parentheses. REA -Reading O Coefficient significantly greater than zero. 

MAT - Mathematics ® Coefficient significantly less than zero. 

SCI -Science 



! j Percentage of Variance Explained 

Source of Variance 

| School Explanatory Models j 

Home 

| School Explanatory with Home Background Control Models 


School 

School 

School Environment 

Background 

School 

School 

School Environment 


Environment 

Instruction 

and Instruction 

Control Model 

Environment 

Instruction 

and Instruction 


Reading 


Between Schools (32%) 

37 

0 

38 

74 

74 

76 

76 

Within Schools (68%) 

— 

— 

— 

20 

20 

20 

20 

Total 

12 

0 

12 

37 

37 

38 

38 

Mathematics 

Between Schools (36%) 

43 

0 

43 

77 

77 

78 

79 

Within Schools (64%) 

— 

— 

— 

22 

22 

22 

22 

Total 

15 

0 

15 

42 

42 

42 

42 

Science 

Between Schools (38%) 

37 

0 

38 

68 

69 

70 

70 

Within Schools (62%) 

— 

— 

— 

21 

21 

21 

21 

Total 

14 

0 

15 

39 

39 

40 

40 


( ) Percentage of available variance shown in parentheses. 
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Exhibit 3.16: School Effectiveness Models for Reading, Mathematics, and Science - 
Iran, Islamic Republic of 


HLM Regression Coefficients 


TIMSS &PIRLSI 
201ll 



School Explanatory Models 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 

School Environment 



REA 

0 (4.4) 

— 

0 (4.3) 

— 

6 (3.5) 

— 

6 (3.6) 

Schools Are Safe and Orderly 

MAT 

-1 (4.8) 

— 

-1 (4.8) 

— 

5 (3.8) 

— 

4 (3.8) 


SCI 

-2 (5.3) 

— 

-1 (5.2) 

— 

5 (4.1) 

— 

5 (4.2) 


REA 

14(3.7) O 

— 

12(3.6) O 

— 

3 (3.1) 

— 

3 (3.1) 

Schools Support Academic Success 

MAT 

13 (3.9) O 

— 

11 (3.9) O 

— 

2 (3.2) 

— 

2 (3.2) 


SCI 

15 (4.1) O 

— 

13 (4.1) O 

— 

3 (3.3) 

— 

3 (3.3) 


REA 

2 (3.7) 

— 

3 (3.7) 

— 

-1 (2.8) 

— 

-1 (2.8) 

Adequate Environment and Resources 

MAT 

2 (3.7) 

— 

3 (3.7) 

— 

-2 (2.7) 

— 

-2 (2.8) 


SCI 

1 (4.3) 

— 

2 (4.1) 

— 

-2 (3.1) 

— 

-2 (3.2) 


School Instruction 



REA 

12 (2.6) O 

10 (2.6) O 

— 


— 


3 (1.9) 

2 (1.9) 


Early Emphasis on Reading Skills 

MAT 

9(2.5) O 

7(2.5) O 

— 


— 


1 (1.9) 

0 (1.9) 



SCI 

11 (2.9) O 

9(2.8) O 

— 


— 


1 (2.1) 

0 (2.2) 


Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

3 (5.8) 

0 (5.9) 

— 


— 


5 (4.4) 

3 (4.5) 


MAT 

3 (5.8) 

0 (6.0) 

— 


— 


5 (4.5) 

3 (4.6) 


SCI 

0 (6.6) 

-3 (6.6) 

— 


— 


2 (4.7) 

0 (4.7) 


Home Background Control Variables 










Students within Schools 












REA 

— 

— 

8 (0.9) 

O 

8 (0.9) 

O 

8 (0.9) O 

8 (0.9) 

O 

Home Resources for Learning 

MAT 

— 

— 

7 (1.1) 

O 

7 (1.1) 

O 

7 (1.1) O 

7 (1.1) 

O 


SCI 

— 

— 

8 (1.1) 

O 

8 (1.1) 

O 

8 (1.1) O 

8 (1.1) 

O 


REA 

— 

— 

8 (0.7) 

O 

8 (0.7) 

O 

8(0.7) O 

8 (0.7) 

O 

Early Literacy/Numeracy Tasks 

MAT 

— 

— 

9 (0.7) 

O 

9 (0.7) 

O 

9(0.7) O 

9 (0.7) 

O 


SCI 

— 

— 

8 (0.7) 

O 

8 (0.7) 

O 

8(0.7) O 

8 (0.7) 

O 

Between Schools 

School Average of 
Home Resources for Learning 

REA 

— 

— 

24 (2.0) 

O 

23 (2.4) 

O 

23 (2.0) O 

22 (2.4) 

O 

MAT 

— 

— 

23 (2.2) 

O 

22 (2.5) 

O 

23 (2.4) O 

23 (2.6) 

O 

SCI 

— 

— 

26 (2.2) 

O 

26 (2.5) 

O 

26 (2.4) O 

26 (2.6) 

O 

School Average of 

Early Literacy/Numeracy Tasks 

REA 

— 

— 

4 (4.8) 


4 (4.9) 


3 (4.7) 

3 (4.9) 


MAT 

— 

— 

0 (4.9) 


0 (5.0) 


0 (4.9) 

0 (5.1) 


SCI 

— 

— 

4 (5.3) 


4 (5.5) 


4 (5.4) 

4 (5.6) 


( ) Standard errors appear in parentheses. 

REA - Reading 

O Coefficient significantly greater than zero. 









MAT - Mathematics 

® Coefficient significantly less than zero. 









SCI -Science 












Percentage of Variance Explained 


School Explanatory Models ^ome School Explanatory with Home Background Control Models 

School School School Environment Background School School School Environment 

Environment Instruction and Instruction Control Model Environment Instruction and Instruction 


Reading 

Between Schools (39%) 

13 

10 

19 

57 

59 

58 

59 

Within Schools (61%) 

— 

— 

— 

7 

7 

7 

7 

Total 

5 

4 

7 

26 

27 

27 

27 

Mathematics 

Between Schools (38%) 

10 

6 

14 

51 

53 

52 

53 

Within Schools (62%) 

— 

— 

— 

8 

8 

8 

8 

Total 

4 

2 

5 

25 

25 

25 

25 

Science 

Between Schools (42%) 

10 

7 

15 

55 

56 

55 

56 

Within Schools (58%) 

— 

— 

— 

7 

7 

7 

7 

Total 

4 

3 

6 

27 

28 

27 

28 


( ) Percentage of available variance shown in parentheses. 
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Exhibit 3.17: School Effectiveness Models for Reading, Mathematics, and Science - Ireland 2011 



HLM Regression Coefficients 


School Explanatory Models 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 

School Environment 



REA 

11 (4.0) 

o 

— 

11 (3.9) 

o 

— 


7(3.1) © 

— 


5 (3.2) 


Schools Are Safe and Orderly 

MAT 

12 (3.8) 

o 

— 

11 (3.8) 

o 

— 


6 (3.9) 

— 


4 (3.8) 



SCI 

11 (4.2) 

o 

— 

11 (4.1) 

o 

— 


5 (4.3) 

— 


3 (4.2) 



REA 

5 (2.2) 

o 

— 

5 (2.2) 

o 

— 


2 (1.9) 

— 


2 (1.9) 


Schools Support Academic Success 

MAT 

5 (2.2) 

o 

— 

6 (2.4) 

o 

— 


3 (2.1) 

— 


4 (2.2) 



SCI 

6 (2.4) 

o 

— 

7 (2.5) 

o 

— 


4 (2.3) 

— 


5 (2.4) 

o 


REA 

-1 (2.1) 


— 

-2 (2.1) 


— 


-1 (1.5) 

— 


-1 (1.6) 


Adequate Environment and Resources 

MAT 

0 (2.5) 


— 

-1 (2.3) 


— 


1 (2.1) 

— 


0 (2.0) 



SCI 

0 (2.9) 


— 

-1 (2.9) 


— 


1 (2.5) 

— 


0 (2.5) 


School Instruction 


REA 

— 


3 (1.7) 

2 (1.8) 


— 


— 

2 (1.5) 


2 (1.4) 


Early Emphasis on Reading Skills 

MAT 

— 


1 (1.7) 

0 (1.7) 


— 


— 

1 (1.6) 


1 (1.5) 



SCI 

— 


3 (1.8) 

2 (1.9) 


— 


— 

2 (1.8) 


2 (1.6) 


Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 


6 (4.4) 

5 (4.4) 


— 


— 

7 (3.3) 

o 

6 (3.5) 


MAT 

— 


10 (6.6) 

9 (6.5) 


— 


— 

13 (5.2) 

o 

12 (5.5) 

o 

SCI 

— 


9 (6.4) 

9 (6.2) 


— 


— 

11 (5.4) 

o 

11 (5.4) 

o 

Home Background Control Variables 

Students within Schools 


REA 

— 


— 

— 


19 (1.0) 

o 

19 (1.0) O 

19 (1.0) 

o 

19 (1.0) 

o 

Home Resources for Learning 

MAT 

— 


— 

— 


17 (0.9) 

o 

17 (0.9) O 

17 (0.9) 

o 

17 (0.9) 

o 


SCI 

— 


— 

— 


17 (1.3) 

o 

17(1.3) O 

17 (1.3) 

o 

17 (1.3) 

o 


REA 

— 


— 

— 


8 (1.2) 

o 

8 (1.1) O 

8 (1.2) 

o 

8 (1.1) 

o 

Early Literacy/Numeracy Tasks 

MAT 

— 


— 

— 


10 (1.3) 

o 

10(1.3) O 

10 (1.3) 

o 

10 (1.3) 

o 


SCI 

— 


— 

— 


7 (1.0) 

o 

7 (1.0) O 

7 (1.0) 

o 

7 (1.0) 

o 

Between Schools 

School Average of 
Home Resources for Learning 

REA 

— 


— 

— 


27 (3.1) 

o 

23 (3.1) O 

27 (3.2) 

o 

24 (3.1) 

o 

MAT 

— 


— 

— 


25 (3.8) 

o 

20 (4.0) O 

25 (3.7) 

o 

21 (3.8) 

o 

SCI 

— 


— 

— 


27 (3.6) 

o 

22 (3.8) O 

27 (3.7) 

o 

23 (3.6) 

o 

School Average of 

Early Literacy/Numeracy Tasks 

REA 

— 


— 

— 


2 (7.5) 


5 (7.0) 

0 (7.6) 


3 (7.1) 


MAT 

— 


— 

— 


-2 (7.1) 


1 (6.9) 

-5 (7.6) 


-3 (7.4) 


SCI 

— 


— 

— 


-1 (7.6) 


1 (7.0) 

-5 (8.0) 


-4 (7.4) 



() Standard errors appear in parentheses. REA -Reading © Coefficient significantlygreaterthan zero. 


MAT 

SCI 

-Mathematics ® Coefficient significantly less than zero. 
- Science 







Percentage of Variance Explained 

Source of Variance 


School Explanatory Models 


Home 


School Explanatory with Home Background Control Models 



School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


Background 
Control Model 


School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Reading 








Between Schools (12%) 

37 

4 

38 

57 

68 

61 

70 

Within Schools (88%) 

— 

— 

— 

20 

20 

20 

20 

Total 

4 

0 

4 

25 

26 

25 

26 

Mathematics 








Between Schools (17%) 

31 

4 

33 

37 

46 

43 

51 

Within Schools (83%) 

— 

— 

— 

21 

21 

21 

21 

Total 

5 

1 

6 

24 

25 

25 

26 

Science 








Between Schools (19%) 

29 

4 

32 

40 

49 

46 

54 

Within Schools (81%) 

— 

— 

— 

19 

19 

19 

19 

Total 

5 

1 

6 

23 

25 

24 

25 


( ) Percentage of available variance shown in parentheses. 
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Exhibit 3.18: School Effectiveness Models for Reading, Mathematics, and Science - Italy 


TIMSS & PIRLS 
2011 


4 th 

Grade 



| HLM Regression Coefficients 

Variables 

| School Explanatory Models | 

Home 

| School Explanatory with Home Background Control Models 


School 

School 

School Environment 

Background 

School 

School 

School Environment 


Environment 

Instruction 

and Instruction 

Control Model 

Environment 

Instruction 

and Instruction 


School Explanatory Variables 

School Environment 



REA 

7(2.6) O 

— 

7(2.6) O 

— 

5(2.6) O 

— 

5 (2.6) 

Schools Are Safe and Orderly 

MAT 

7 (4.4) 

— 

7 (4.4) 

— 

5 (4.3) 

— 

5 (4.3) 


SCI 

9(3.9) O 

— 

9(3.9) O 

— 

7 (4.0) 

— 

7 (4.0) 


REA 

-3 (2.5) 

— 

-2 (2.6) 

— 

-3 (2.2) 

— 

-3 (2.2) 

Schools Support Academic Success 

MAT 

-1 (3.1) 

— 

-1 (3.3) 

— 

-2 (3.1) 

— 

-2 (3.2) 


SCI 

-2 (3.1) 

— 

-2 (3.2) 

— 

-2 (2.9) 

— 

-2 (3.1) 


REA 

3 (2.4) 

— 

3 (2.5) 

— 

3 (2.0) 

— 

3 (2.0) 

Adequate Environment and Resources 

MAT 

6(3.2) O 

— 

6(3.2) O 

— 

7(3.0) O 

— 

7(3.1) O 


SCI 

4 (3.2) 

— 

4 (3.2) 

— 

4 (2.8) 

— 

4 (2.8) 



REA 

— 

-1 (1.6) 

-1 (1.6) 

— 

— 

0 (1.5) 

0 (1.5) 

Early Emphasis on Reading Skills 

MAT 

— 

0 (2.0) 

-1 (2.1) 

— 

— 

1 (2.0) 

0 (2.2) 


SCI 

— 

-1 (2.1) 

-2 (2.2) 

— 

— 

0 (2.0) 

-1 (2.1) 

Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 

4 (3.8) 

3 (4.1) 

— 

— 

6 (4.0) 

5 (4.1) 

MAT 

— 

6 (5.8) 

3 (5.6) 

— 

— 

6 (5.9) 

4 (5.4) 

SCI 

— 

4 (5.7) 

2 (5.6) 

— 

— 

6 (5.8) 

4 (5.5) 


Home Background Control Variables 

Students within Schools 



REA 

— 

— 

— 

16 (0.9) 

o 

16 (0.9) 

o 

16 (0.9) 

o 

16 (0.9) 

o 

Home Resources for Learning 

MAT 

— 

— 

— 

12 (1.0) 

o 

12 (1.0) 

o 

12 (1.0) 

o 

12 (1.0) 

o 


SCI 

— 

— 

— 

15 (1.0) 

o 

15 (1.0) 

o 

15 (1.0) 

o 

15 (1.0) 

o 


REA 

— 

— 

— 

7 (0.9) 

o 

7 (0.9) 

o 

7 (0.9) 

o 

7 (0.9) 

o 

Early Literacy/Numeracy Tasks 

MAT 

— 

— 

— 

10 (0.9) 

o 

10 (0.9) 

o 

10 (0.9) 

o 

10 (0.9) 

o 


SCI 

— 

— 

— 

6 (0.9) 

o 

6 (0.9) 

o 

6 (0.9) 

o 

6 (0.9) 

o 

Between Schools 

School Average of 
Home Resources for Learning 

REA 

— 

— 

— 

22 (3.5) 

o 

21 (3.4) 

o 

22 (3.5) 

o 

21 (3.4) 

o 

MAT 

— 

— 

— 

20 (4.6) 

o 

19 (4.6) 

o 

20 (4.5) 

o 

19 (4.5) 

o 

SCI 

— 

— 

— 

23 (4.4) 

o 

22 (4.3) 

o 

23 (4.4) 

o 

22 (4.3) 

o 

School Average of 

Early Literacy/Numeracy Tasks 

REA 

— 

— 

— 

2 (5.3) 


3 (5.3) 


0 (5.4) 


2 (5.4) 


MAT 

— 

— 

— 

7 (6.5) 


8 (6.7) 


4 (6.6) 


7 (6.8) 


SCI 

— 

— 

— 

1 (6.2) 


2 (6.4) 


-1 (6.3) 


1 (6.4) 



() Standard errors appear in parentheses. REA -Reading O Coefficient significantly greater than zero. 

MAT - Mathematics ® Coefficient significantly less than zero. 

SCI -Science 



| Percentage of Variance Explained 

Source of Variance 

| School Explanatory Models jlj 

Home 

| School Explanatory with Home Background Control Models 


School 

School 

School Environment 

Background 

School 

School 

School Environment 


Environment 

Instruction 

and Instruction 

Control Model 

Environment 

Instruction 

and Instruction 


Reading 


Between Schools (17%) 

6 

1 

7 

22 

26 

23 

26 

Within Schools (83%) 

— 

— 

— 

15 

15 

15 

15 

Total 

1 

0 

1 

16 

17 

16 

17 

Mathematics 

Between Schools (26%) 

7 

1 

7 

8 

13 

9 

14 

Within Schools (74%) 

— 

— 

— 

15 

15 

15 

15 

Total 

2 

0 

2 

13 

14 

13 

14 

Science 

Between Schools (26%) 

5 

0 

6 

14 

18 

15 

18 

Within Schools (74%) 

— 

— 

— 

14 

14 

14 

14 

Total 

1 

0 

2 

14 

15 

14 

15 


( ) Percentage of available variance shown in parentheses. 
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TIMSS ' PIRLSI 

Exhibit 3.19: School Effectiveness Models for Reading, Mathematics, and Science - Lithuania 2011 



HLM Regression Coefficients 


School Explanatory Models 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 

School Environment 



REA 

8 (6.0) 


— 


7 (5.5) 


— 


7 (3.3) 

o 

— 


7 (3.4) 

o 

Schools Are Safe and Orderly 

MAT 

6 (6.5) 


— 


5 (6.3) 


— 


5 (3.7) 


— 


6 (3.8) 



SCI 

8 (6.5) 


— 


7 (5.8) 


— 


7 (2.9) 

o 

— 


7 (2.9) 

o 


REA 

11 (2.8) 

o 

— 


9 (2.6) 

o 

— 


0 (1.9) 


— 


0 (1.8) 


Schools Support Academic Success 

MAT 

13 (3.4) 

o 

— 


12 (3.2) 

o 

— 


1 (2.0) 


— 


1 (2.0) 



SCI 

10 (3.5) 

o 

— 


9 (3.3) 

o 

— 


-1 (2.1) 


— 


-1 (2.1) 



REA 

-4 (3.9) 


— 


-4 (4.0) 


— 


2 (1.8) 


— 


1 (1.8) 


Adequate Environment and Resources 

MAT 

-5 (4.6) 


— 


-5 (4.6) 


— 


2 (2.2) 


— 


2 (2.2) 



SCI 

-5 (4.8) 


— 


-5 (4.8) 


— 


1 (2.2) 


— 


1 (2.2) 


School Instruction 


REA 

— 


5 (1.8) 

o 

4 (1.9) 

o 

— 


— 


2 (1.2) 


2 (1.2) 


Early Emphasis on Reading Skills 

MAT 

— 


4 (2.1) 

o 

3 (1.9) 


— 


— 


2 (1.1) 


2 (1.1) 



SCI 

— 


3 (2.0) 


3 (2.0) 


— 


— 


1 (1.2) 


2 (1.2) 


Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 


18 (6.1) 

o 

13 (5.0) 

o 

— 


— 


8 (3.1) 

o 

6 (3.1) 


MAT 

— 


14 (6.3) 

o 

9 (5.3) 


— 


— 


3 (3.6) 


1 (3.5) 


SCI 

— 


18 (6.2) 

o 

13 (4.8) 

o 

— 


— 


7 (3.2) 

o 

5 (3.1) 


Home Background Control Variables 

Students within Schools 


REA 

— 


— 


— 


13 (0.8) 

o 

13 (0.8) 

o 

13 (0.8) 

o 

13 (0.8) 

o 

Home Resources for Learning 

MAT 

— 


— 


— 


11 (1.2) 

o 

11 (1.2) 

o 

11 (1.2) 

o 

11 (1.2) 

o 


SCI 

— 


— 


— 


10 (0.9) 

o 

10 (0.9) 

o 

10 (0.9) 

o 

10 (0.9) 

o 


REA 

— 


— 


— 


15 (0.9) 

o 

15 (0.9) 

o 

15 (0.9) 

o 

15 (0.9) 

o 

Early Literacy/Numeracy Tasks 

MAT 

— 


— 


— 


18 (0.9) 

o 

18 (0.9) 

o 

18 (0.9) 

o 

18 (0.9) 

o 


SCI 

— 


— 


— 


14 (0.9) 

o 

14 (0.9) 

o 

14 (0.9) 

o 

14 (0.9) 

o 

Between Schools 

School Average of 
Home Resources for Learning 

REA 

— 


— 


— 


19 (3.4) 

o 

19 (3.1) 

o 

18 (3.0) 

o 

19 (2.8) 

o 

MAT 

— 


— 


— 


21 (3.7) 

o 

21 (3.6) 

o 

21 (3.6) 

o 

21 (3.4) 

o 

SCI 

— 


— 


— 


18 (3.4) 

o 

19 (3.3) 

o 

18 (3.2) 

o 

18 (3.1) 

o 

School Average of 

Early Literacy/Numeracy Tasks 

REA 

— 


— 


— 


26 (6.6) 

o 

25 (5.8) 

o 

24 (6.1) 

o 

23 (5.2) 

o 

MAT 

— 


— 


— 


27 (5.4) 

o 

26 (5.4) 

o 

26 (5.5) 

o 

26 (5.3) 

o 

SCI 

— 


— 


— 


29 (5.7) 

o 

28 (5.5) 

o 

27 (5.3) 

o 

27 (5.1) 

o 


() Standard errors appear in parentheses. REA - Reading O Coefficient significantly greaterthan zero. 

MAT - Mathematics ® Coefficient significantly less than zero. 

SCI -Science 


Source of Variance 

|| Percentage of Variance Explained 

| School Explanatory Models | 

Home 

| School Explanatory with Home Background Control Models 


School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


Reading 


Between Schools (19%) 

24 

15 

31 

78 

82 

80 

83 

Within Schools (81%) 

— 

— 

— 

21 

21 

21 

21 

Total 

5 

3 

6 

32 

33 

33 

33 

Mathematics 

Between Schools (20%) 

23 

9 

27 

78 

81 

78 

81 

Within Schools (80%) 

— 

— 

— 

22 

22 

22 

22 

Total 

5 

2 

5 

33 

34 

33 

34 

Science 

Between Schools (23%) 

20 

12 

26 

77 

80 

78 

81 

Within Schools (77%) 

— 

— 

— 

20 

20 

20 

20 

Total 

5 

3 

6 

33 

34 

33 

34 


( ) Percentage of available variance shown in parentheses. 
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Exhibit 3.20: School Effectiveness Models for Reading, Mathematics, and Science - Malta 


TIMSS & PIRLS 
2011 



HLM Regression Coefficients 


School Explanatory Models | 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 

School Environment 



REA 

21 (5.7) 

o 

— 

18 (5.6) O 

— 


11 (4.9) 

o 

— 


8 (4.1) 

o 

Schools Are Safe and Orderly 

MAT 

15 (3.6) 

o 

— 

14 (3.6) O 

— 


10 (3.2) 

o 

— 


9 (3.0) 

o 


SCI 

16 (4.6) 

o 

— 

14 (4.6) O 

— 


7 (4.0) 


— 


6 (3.6) 



REA 

21 (4.2) 

o 

— 

19 (4.1) O 

— 


8 (3.1) 

o 

— 


4 (2.9) 


Schools Support Academic Success 

MAT 

12 (2.7) 

o 

— 

11 (2.7) O 

— 


6 (2.5) 

o 

— 


5 (2.6) 



SCI 

18 (3.8) 

o 

— 

16 (3.8) O 

— 


6 (2.9) 

o 

— 


4 (2.9) 



REA 

0 (4.6) 


— 

-1 (4.5) 

— 


-3 (2.8) 


— 


-3 (2.7) 


Adequate Environment and Resources 

MAT 

-1 (2.9) 


— 

-1 (2.8) 

— 


-2 (2.2) 


— 


-2 (2.1) 



SCI 

2 (3.9) 


— 

2 (3.9) 

— 


0 (2.6) 


— 


0 (2.5) 


School Instruction 


REA 

— 


-2 (3.2) 

-3 (2.7) 

— 


— 


-2 (1.8) 


-2 (1.7) 


Early Emphasis on Reading Skills 

MAT 

— 


-1 (2.2) 

-3 (1.9) 

— 


— 


-1 (1.6) 


-2 (1.5) 



SCI 

— 


-1 (2.8) 

-3 (2.4) 

— 


— 


-2 (1.7) 


-2 (1.7) 


Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 


47 (9.7) O 

22 (8.1) O 

— 


— 


30 (6.9) 

o 

24 (6.3) 

o 

MAT 

— 


27 (6.8) O 

10 (5.9) 

— 


— 


17 (5.7) 

o 

10 (5.5) 


SCI 

— 


36 (8.2) O 

15 (7.1) O 

— 


— 


21 (6.1) 

o 

16 (6.1) 

o 

Home Background Control Variables 













Students within Schools 















REA 

— 


— 

— 

24 (1.4) 

o 

24 (1.4) 

o 

24 (1.4) 

o 

24 (1.4) 

o 

Home Resources for Learning 

MAT 

— 


— 

— 

15 (1.1) 

o 

15 (1.1) 

o 

15 (1.1) 

o 

15 (1.1) 

o 


SCI 

— 


— 

— 

23 (1.6) 

o 

23 (1.6) 

o 

23 (1.6) 

o 

23 (1.6) 

o 


REA 

— 


— 

— 

12 (1.2) 

o 

12 (1.2) 

o 

12 (1.2) 

o 

12 (1.2) 

o 

Early Literacy/Numeracy Tasks 

MAT 

— 


— 

— 

10 (1.0) 

o 

10 (1.0) 

o 

10 (1.0) 

o 

10 (1.0) 

o 


SCI 

— 


— 

— 

8 (1.2) 

o 

8 (1.2) 

o 

8 (1.2) 

o 

8 (1.2) 

o 

Between Schools 

School Average of 
Home Resources for Learning 

REA 

— 


— 

— 

65 (5.8) 

o 

52 (6.9) 

o 

64 (5.1) 

o 

57 (6.3) 

o 

MAT 

— 


— 

— 

34 (4.5) 

o 

24 (5.0) 

o 

34 (4.1) 

o 

25 (4.9) 

o 

SCI 

— 


— 

— 

55 (5.5) 

o 

45 (6.2) 

o 

55 (4.8) 

o 

48 (5.8) 

o 

School Average of 

Early Literacy/Numeracy Tasks 

REA 

— 


— 

— 

41 (9.2) 

o 

35 (9.4) 

o 

22 (9.9) 

o 

22 (9.9) 

o 

MAT 

— 


— 

— 

27 (8.0) 

o 

21 (8.2) 

o 

16 (8.6) 


16 (8.8) 


SCI 

— 


— 

— 

32 (9.7) 

o 

28 (9.6) 

o 

19 (10.2) 


20 (10.3) 


( ) Standard errors appear in parentheses. 

REA - 

Reading 

O Coefficient significantly greater than zero. 










MAT- 

Mathematics 

® Coefficient significantly less than zero. 










SCI - 

Science 














Percentage of Variance Explained 


School Explanatory Models H ome School Explanatory with Home Background Control Models 

School School School Environment Background School School School Environment 

Environment Instruction and Instruction Control Model Environment Instruction and Instruction 


Reading 

Between Schools (28%) 

48 

25 

52 

75 

81 

83 

85 

Within Schools (72%) 

— 

— 

— 

13 

13 

13 

13 

Total 

13 

7 

14 

30 

32 

33 

33 

Mathematics 

Between Schools (20%) 

49 

21 

53 

57 

69 

64 

71 

Within Schools (80%) 

— 

— 

— 

10 

10 

10 

10 

Total 

10 

4 

10 

19 

22 

21 

22 

Science 

Between Schools (24%) 

44 

20 

48 

74 

78 

80 

81 

Within Schools (76%) 

— 

— 

— 

13 

13 

13 

13 

Total 

11 

5 

12 

28 

29 

29 

30 


( ) Percentage of available variance shown in parentheses. 
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TIMSS ft-. PIRLSI 

Exhibit 3.21: School Effectiveness Models for Reading, Mathematics, and Science - Morocco 2011 1 



HLM Regression Coefficients 


School Explanatory Models 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 

School Environment 



REA 

5 (5.0) 


— 


1 (4.7) 


— 

4 (4.6) 

— 


0 (4.3) 


Schools Are Safe and Orderly 

MAT 

4 (5.0) 


— 


0 (4.8) 


— 

5 (4.8) 

— 


1 (4.7) 



SCI 

5 (5.9) 


— 


0 (5.6) 


— 

5 (5.9) 

— 


1 (5.7) 



REA 

18 (3.7) 

o 

— 


12 (3.7) 

o 

— 

13 (4.2) O 

— 


8 (3.9) 


Schools Support Academic Success 

MAT 

12 (4.1) 

o 

— 


7 (4.2) 


— 

12 (4.9) © 

— 


8 (4.6) 



SCI 

17 (4.6) 

o 

— 


11 (4.4) 

o 

— 

15 (5.8) O 

— 


10 (5.4) 



REA 

8 (3.7) 

o 

— 


10 (3.5) 

o 

— 

10 (3.5) O 

— 


11 (3.2) 

o 

Adequate Environment and Resources 

MAT 

10 (3.8) 

o 

— 


11 (3.5) 

o 

— 

9(3.9) O 

— 


10 (3.6) 

o 


SCI 

10 (4.3) 

o 

— 


11 (3.8) 

o 

— 

10 (4.4) O 

— 


11 (3.9) 

o 

School Instruction 


REA 

— 


11 (3.2) 

o 

7 (3.6) 


— 

— 

8 (3.5) 

o 

6 (3.7) 


Early Emphasis on Reading Skills 

MAT 

— 


5 (3.3) 


3 (3.9) 


— 

— 

5 (3.6) 


4 (3.9) 



SCI 

— 


7 (3.4) 

o 

4 (4.0) 


— 

— 

6 (3.7) 


4 (4.0) 


Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 


30 (6.6) 

o 

26 (6.6) 

o 

— 

— 

27 (6.5) 

o 

24 (6.5) 

o 

MAT 

— 


28 (7.0) 

o 

25 (6.9) 

o 

— 

— 

27 (6.9) 

o 

24 (6.7) 

o 

SCI 

— 


34 (7.7) 

o 

30 (7.6) 

o 

— 

— 

33 (7.5) 

o 

30 (7.3) 

o 

Home Background Control Variables 

Students within Schools 


REA 

— 


— 


— 


1 (1.1) 

1 (1.1) 

1 (1-1) 


1 (1.1) 


Home Resources for Learning 

MAT 

— 


— 


— 


0 (1.2) 

0 (1.2) 

0 (1.2) 


0 (1.2) 



SCI 

— 


— 


— 


1 (1.5) 

1 (1.5) 

1 (1.5) 


1 (1.5) 



REA 

— 


— 


— 


14 (1.5) o 

14 (1.5) O 

14 (1.5) 

o 

14 (1.5) 

o 

Early Literacy/Numeracy Tasks 

MAT 

— 


— 


— 


10 (1.4) o 

10 (1.4) O 

10 (1.4) 

o 

10 (1.4) 

o 


SCI 

— 


— 


— 


15 (1.8) O 

15 (1.8) O 

15 (1.8) 

o 

15 (1.8) 

o 

Between Schools 

School Average of 
Home Resources for Learning 

REA 

— 


— 


— 


17 (8.4) O 

8 (8.5) 

14 (7.2) 


8 (7.3) 


MAT 

— 


— 


— 


8 (8.6) 

0 (8.8) 

6 (7.4) 


1 (7.8) 


SCI 

— 


— 


— 


12 (9.6) 

2 (10.1) 

9 (8.0) 


3 (8.8) 


School Average of 

Early Literacy/Numeracy Tasks 

REA 

— 


— 


— 


8 (6.4) 

10 (6.1) 

5 (5.4) 


7 (5.3) 


MAT 

— 


— 


— 


-5 (7.2) 

-3 (6.9) 

-8 (6.1) 


-6 (6.1) 


SCI 

— 


— 


— 


-3 (8.5) 

-1 (8.1) 

-7 (7.0) 


-4 (7.0) 



() Standard errors appear in parentheses. REA -Reading © Coefficient significantlygreaterthan zero. 


MAT 

SCI 

-Mathematics ® Coefficient significantly less than zero. 
- Science 







Percentage of Variance Explained 

Source of Variance 


School Explanatory Models 


Home 


School Explanatory with Home Background Control Models 



School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


Background 
Control Model 


School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Reading 








Between Schools (38%) 

22 

27 

37 

17 

29 

35 

41 

Within Schools (62%) 

— 

— 

— 

7 

7 

7 

7 

Total 

8 

10 

14 

11 

15 

18 

20 

Mathematics 








Between Schools (44%) 

13 

20 

26 

1 

12 

20 

25 

Within Schools (56%) 

— 

— 

— 

6 

6 

6 

6 

Total 

6 

9 

11 

4 

9 

12 

14 

Science 








Between Schools (36%) 

17 

25 

33 

4 

16 

25 

32 

Within Schools (64%) 

— 

— 

— 

6 

6 

6 

6 

Total 

6 

9 

12 

6 

10 

13 

15 


( ) Percentage of available variance shown in parentheses. 
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Exhibit 3.22: School Effectiveness Models for Reading, Mathematics, and Science - 
Northern Ireland 


HLM Regression Coefficients 


TIMSS &PIRLSI 
201ll 



School Explanatory Models | 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 

School Environment 



REA 

14 (4.0) 

O 

— 

14 (3.6) O 

— 

10 (3.4) O 

— 

10 (3.1) 

O 

Schools Are Safe and Orderly 

MAT 

20 (5.5) 

O 

— 

20 (5.0) O 

— 

14 (4.6) O 

— 

13 (4.3) 

O 


SCI 

17 (5.4) 

O 

— 

16 (4.9) O 

— 

12(5.1) O 

— 

11 (4.5) 

O 


REA 

5 (2.2) 

O 

— 

5(2.1) O 

— 

3 (1.6) 

— 

2 (1.6) 


Schools Support Academic Success 

MAT 

4 (2.7) 


— 

3 (2.7) 

— 

0 (2.0) 

— 

0 (2.0) 



SCI 

6 (2.3) 

O 

— 

5 (2.3) 

— 

3 (1.9) 

— 

2 (1.9) 



REA 

2 (1.9) 


— 

2 (1.9) 

— 

0 (1.5) 

— 

0 (1.5) 


Adequate Environment and Resources 

MAT 

1 (2.5) 


— 

1 (2.5) 

— 

-1 (1-8) 

— 

-1 (1.9) 



SCI 

0 (2.5) 


— 

0 (2.6) 

— 

-1 (2.1) 

— 

-1 (2.1) 


School Instruction 


REA 

— 


1 (2.4) 

2 (2.1) 

— 

— 

-1 (1.7) 

0 (1.5) 


Early Emphasis on Reading Skills 

MAT 

— 


-1 (3.0) 

1 (2.7) 

— 

— 

-3 (2.4) 

-2 (2.2) 



SCI 

— 


-1 (2.9) 

1 (2.5) 

— 

— 

-2 (2.4) 

-1 (2.2) 


Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 


10 (4.7) O 

6 (4.3) 

— 

— 

3 (4.4) 

1 (3.8) 


MAT 

— 


14 (5.4) O 

10 (4.8) O 

— 

— 

5 (5.1) 

3 (4.4) 


SCI 

— 


14(5.5) O 

9 (5.0) 

— 

— 

6 (5.9) 

4 (5.1) 


Home Background Control Variables 










Students within Schools 












REA 

— 


— 

— 

16(1.1) O 

16(1.1) O 

16(1.1) O 

16 (1.1) 

O 

Home Resources for Learning 

MAT 

— 


— 

— 

16(1.5) O 

16 (1.5) O 

16 (1.5) O 

16 (1.5) 

O 


SCI 

— 


— 

— 

15 (0.9) O 

15 (0.9) O 

15 (0.9) O 

15 (0.9) 

O 


REA 

— 


— 

— 

9(1.4) O 

9(1.4) O 

9(1.4) O 

9 (1.4) 

O 

Early Literacy/Numeracy Tasks 

MAT 

— 


— 

— 

9(1.4) O 

9(1.4) O 

9 (1.4) O 

9 (1.4) 

O 


SCI 

— 


— 

— 

6(1.3) O 

5(1.3) O 

5(1.3) O 

5 (1.3) 

O 

Between Schools 

School Average of 
Home Resources for Learning 

REA 

— 


— 

— 

37(5.2) O 

32(5.0) O 

37(5.3) O 

31 (5.1) 

O 

MAT 

— 


— 

— 

48 (5.9) O 

42(5.7) O 

48 (6.1) O 

43 (5.9) 

O 

SCI 

— 


— 

— 

40 (6.0) O 

34 (5.9) O 

39 (6.3) O 

34 (6.2) 

O 

School Average of 

Early Literacy/Numeracy Tasks 

REA 

— 


— 

— 

7 (8.7) 

10 (7.6) 

6 (10.0) 

9 (8.6) 


MAT 

— 


— 

— 

6 (8.7) 

11 (8.2) 

4 (10.1) 

10 (9.2) 


SCI 

— 


— 

— 

5 (9.2) 

9 (8.5) 

3 (11.1) 

7 (9.7) 


( ) Standard errors appear in parentheses. 

REA - 

Reading 

O Coefficient significantly greater than zero. 







MAT- 

Mathematics 

® Coefficient significantly less than zero. 







SCI - 

Science 











Percentage of Variance Explained 


School Explanatory Models H ome School Explanatory with Home Background Control Models 

School School School Environment Background School School School Environment 

Environment Instruction and Instruction Control Model Environment Instruction and Instruction 


Reading 

Between Schools (11%) 

36 

7 

40 

67 

79 

68 

79 

Within Schools (89%) 

— 

— 

— 

8 

8 

8 

8 

Total 

4 

1 

4 

14 

16 

14 

16 

Mathematics 

Between Schools (17%) 

28 

7 

31 

60 

68 

62 

69 

Within Schools (83%) 

— 

— 

— 

8 

8 

8 

8 

Total 

5 

1 

5 

17 

18 

17 

19 

Science 

Between Schools (22%) 

27 

7 

30 

46 

56 

49 

57 

Within Schools (78%) 

— 

— 

— 

10 

10 

10 

10 

Total 

6 

2 

6 

18 

20 

18 

20 


( ) Percentage of available variance shown in parentheses. 
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TIJVXSS ' PIRLSI 

Exhibit 3.23: School Effectiveness Models for Reading, Mathematics, and Science - Norway " 20111 



HLM Regression Coefficients 


School Explanatory Models j 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 

School Environment 



REA 

7 (3.7) 

— 

7 (3.9) 

— 

7(3.7) O 

— 

7 (3.4) 

o 

Schools Are Safe and Orderly 

MAT 

10 (4.7) O 

— 

11 (5.2) O 

— 

11 (5.0) O 

— 

10 (4.8) 

o 


SCI 

7 (3.8) 

— 

7 (4.0) 

— 

7(3.3) O 

— 

6 (3.2) 



REA 

6(2.3) O 

— 

5 (2.4) O 

— 

3 (2.2) 

— 

2 (2.2) 


Schools Support Academic Success 

MAT 

7 (3.4) 

— 

6 (3.5) 

— 

4 (3.6) 

— 

3 (3.4) 



SCI 

7 (2.4) O 

— 

7(2.5) O 

— 

3 (2.2) 

— 

2 (2.2) 



REA 

-5 (2.5) 

— 

-4 (2.5) 

— 

-2 (2.6) 

— 

-1 (2.4) 


Adequate Environment and Resources 

MAT 

-4 (3.2) 

— 

-4 (3.2) 

— 

-2 (3.4) 

— 

-1 (3.3) 



SCI 

-4 (2.4) 

— 

-4 (2.4) 

— 

-1 (2.2) 

— 

0 (2.1) 


School Instruction 


REA 

— 

-1 (1.7) 

-1 (1.6) 

— 

— 

0 (1.6) 

-1 (1.5) 


Early Emphasis on Reading Skills 

MAT 

— 

0 (2.2) 

-1 (2.0) 

— 

— 

0 (2.1) 

-1 (1.9) 



SCI 

— 

-1 (1.5) 

-1 (1.4) 

— 

— 

0 (1.2) 

-1 (1.1) 


Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 

8 (4.3) 

4 (4.1) 

— 

— 

9(4.2) O 

7 (3.9) 


MAT 

— 

8 (5.7) 

4 (5.7) 

— 

— 

11 (5.6) 

7 (5.4) 


SCI 

— 

6 (4.4) 

2 (4.3) 

— 

— 

8 (4.3) 

5 (4.0) 


Home Background Control Variables 

Students within Schools 


REA 

— 

— 

— 

12(1.1) O 

12(1.1) O 

12(1.1) O 

12 (1.1) 

o 

Home Resources for Learning 

MAT 

— 

— 

— 

9(1.5) O 

9(1.5) O 

9(1.5) O 

9 (1.5) 

o 


SCI 

— 

— 

— 

11 (1.2) O 

11 (1.2) O 

11 (1.2) O 

11 (1.2) 

o 


REA 

— 

— 

— 

12 (0.9) O 

12 (0.9) O 

12 (0.9) O 

12 (0.9) 

o 

Early Literacy/Numeracy Tasks 

MAT 

— 

— 

— 

13 (0.9) O 

13 (0.9) O 

13 (0.9) O 

13 (0.9) 

o 


SCI 

— 

— 

— 

11 (1.0) o 

11 (1.0) o 

11 (1.0) o 

11 (1.0) 

o 

Between Schools 

School Average of 
Home Resources for Learning 

REA 

— 

— 

— 

20 (4.3) O 

16 (4.9) O 

21 (4.1) O 

17 (4.5) 

o 

MAT 

— 

— 

— 

21 (5.6) O 

15(6.7) O 

22(5.3) O 

17 (5.9) 

o 

SCI 

— 

— 

— 

21 (3.5) O 

17(4.3) O 

21 (3.3) O 

18 (3.8) 

o 

School Average of 

Early Literacy/Numeracy Tasks 

REA 

— 

— 

— 

5 (7.8) 

11 (8.0) 

6 (7.4) 

10 (7.4) 


MAT 

— 

— 

— 

7 (10.1) 

15 (10.4) 

8 (9.9) 

15 (10.1) 


SCI 

— 

— 

— 

4 (6.7) 

9 (6.6) 

5 (6.3) 

9 (6.3) 



() Standard errors appear in parentheses. REA -Reading O Coefficient significantly greater than zero. 


MAT 

SCI 

-Mathematics ® Coefficient significantly less than zero. 
- Science 







Percentage of Variance Explained 

Source of Variance 


School Explanatory Models 


Home 


School Explanatory with Home Background Control Models 



School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


Background 
Control Model 


School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Reading 








Between Schools (8%) 

29 

10 

32 

26 

38 

38 

45 

Within Schools (92%) 

— 

— 

— 

19 

19 

19 

19 

Total 

2 

1 

3 

19 

20 

20 

21 

Mathematics 








Between Schools (14%) 

24 

6 

26 

15 

29 

22 

33 

Within Schools (86%) 

— 

— 

— 

18 

18 

18 

18 

Total 

3 

1 

3 

18 

20 

19 

20 

Science 








Between Schools (9%) 

32 

6 

33 

29 

43 

37 

47 

Within Schools (91%) 

— 

— 

— 

21 

21 

21 

21 

Total 

3 

1 

3 

22 

23 

22 

23 


( ) Percentage of available variance shown in parentheses. 
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Exhibit 3.24: School Effectiveness Models for Reading, Mathematics, and Science - Oman 


TIMSS & PIRLS 
2011 


4 th 

Grade 



| HLM Regression Coefficients 

Variables 

| School Explanatory Models j 

Home 

| School Explanatory with Home Background Control Models 


School 

School 

School Environment 

Background 

School 

School 

School Environment 


Environment 

Instruction 

and Instruction 

Control Model 

Environment 

Instruction 

and Instruction 


School Explanatory Variables 

School Environment 



REA 

7 (3.7) 


— 


5 (3.5) 


— 

11 (3.3) 

o 

— 

9 (3.3) 

o 

Schools Are Safe and Orderly 

MAT 

8 (3.6) 

o 

— 


6 (3.5) 


— 

11 (3.4) 

o 

— 

8 (3.4) 

o 


SCI 

10 (4.3) 

o 

— 


7 (4.0) 


— 

14 (4.0) 

o 

— 

9 (4.0) 

o 


REA 

12 (2.7) 

o 

— 


9 (2.6) 

o 

— 

8 (2.5) 

o 

— 

7 (2.5) 

o 

Schools Support Academic Success 

MAT 

12 (2.7) 

o 

— 


9 (2.7) 

o 

— 

8 (2.6) 

o 

— 

6 (2.6) 

o 


SCI 

15 (3.1) 

o 

— 


11 (3.1) 

o 

— 

11 (2.9) 

o 

— 

8 (3.0) 

o 


REA 

3 (2.9) 


— 


4 (2.8) 


— 

0 (2.7) 


— 

1 (2.7) 


Adequate Environment and Resources 

MAT 

-1 (2.9) 


— 


1 (2.8) 


— 

-3 (2.8) 


— 

-1 (2.7) 



SCI 

-2 (3.6) 


— 


0 (3.4) 


— 

-4 (3.4) 


— 

-2 (3.2) 


School Instruction 


REA 

— 


5 (2.0) 

o 

4 (1.9) 


— 

— 


2 (1.7) 

2 (1.7) 


Early Emphasis on Reading Skills 

MAT 

— 


4 (2.0) 

o 

3 (1.9) 


— 

— 


2 (1.8) 

2 (1.8) 



SCI 

— 


5 (2.3) 

o 

3 (2.3) 


— 

— 


2 (2.1) 

2 (2.1) 


Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 


22 (4.2) 

o 

19 (4.3) 

o 

— 

— 


21 (4.4) O 

17 (4.5) 

o 

MAT 

— 


24 (4.2) 

o 

20 (4.5) 

o 

— 

— 


23 (4.4) O 

19 (4.7) 

o 

SCI 

— 


34 (5.3) 

o 

30 (5.6) 

o 

— 

— 


33 (5.5) O 

29 (5.9) 

o 


Home Background Control Variables 

Students within Schools 



REA 

— 

— 

— 

12 (1.0) O 

12 (1.0) O 

12 (1.0) 

o 

12 ( 1 . 0 ) o 

Home Resources for Learning 

MAT 

— 

— 

— 

13 (0.9) O 

13 (0.9) O 

13 (0.9) 

o 

13 (0.9) o 


SCI 

— 

— 

— 

15(1.3) O 

15(1.3) O 

15 (1.3) 

o 

15 (1.3) o 


REA 

— 

— 

— 

19 (1.0) O 

19 (1.0) O 

19 (1.0) 

o 

19 (1.0) o 

Early Literacy/Numeracy Tasks 

MAT 

— 

— 

— 

18 (0.9) O 

18 (0.9) O 

18 (0.9) 

o 

18 (0.9) o 


SCI 

— 

— 

— 

22(1.2) O 

22(1.2) O 

22 (1.2) 

o 

22(1.2) O 

Between Schools 

School Average of 
Home Resources for Learning 

REA 

— 

— 

— 

22(3.5) O 

19(3.5) O 

19 (3.5) 

o 

18 (3.5) O 

MAT 

— 

— 

— 

18 (3.9) O 

16 (3.8) O 

15 (3.7) 

o 

14 (3.8) O 

SCI 

— 

— 

— 

18 (4.6) O 

17 (4.4) O 

16 (4.4) 

o 

14 (4.4) O 

School Average of 

Early Literacy/Numeracy Tasks 

REA 

— 

— 

— 

-9 (13.9) 

-6 (13.1) 

-10 (11.6) 


-7 (11.4) 

MAT 

— 

— 

— 

-5 (11.7) 

-4 (11.0) 

-7 (9.3) 


-5 (9.1) 

SCI 

— 

— 

— 

-6 (15.2) 

-4 (14.2) 

-7 (11.4) 


-6 (11.2) 


() Standard errors appear in parentheses. REA -Reading O Coefficient significantly greater than zero. 


MAT 

-Mathematics ® Coefficient significantly less than zero. 





SCI 

-Science 









Percentage of Variance Explained 

Source of Variance 


School Explanatory Models 


Home 


School Explanatory with Home Background Control Models 



School 

School 

School Environment 


Background 


School 

School 

School Environment 



Environment 

Instruction 

and Instruction 


Control Model 


Environment 

Instruction 

and Instruction 

Reading 








Between Schools (18%) 

18 

16 

28 

17 

29 

28 

36 

Within Schools (82%) 

— 

— 

— 

15 

15 

15 

15 

Total 

3 

3 

5 

15 

18 

17 

19 

Mathematics 








Between Schools (20%) 

13 

16 

24 

10 

20 

22 

28 

Within Schools (80%) 

— 

— 

— 

16 

16 

16 

16 

Total 

3 

3 

5 

15 

17 

17 

18 

Science 








Between Schools (21%) 

14 

21 

28 

7 

19 

24 

30 

Within Schools (79%) 

— 

— 

— 

17 

17 

17 

17 

Total 

3 

4 

6 

15 

17 

18 

20 


( ) Percentage of available variance shown in parentheses. 
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Exhibit 3.25: School Effectiveness Models for Reading, Mathematics, and Science - Poland 


HLM Regression Coefficients 


TIMSS &PIRLSI 
201l! 



School Explanatory Models 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 

School Environment 



REA 

-7 (4.5) 


— 

-5 (4.4) 

— 


3 (3.6) 

— 

2 (3.4) 


Schools Are Safe and Orderly 

MAT 

-8 (5.0) 


— 

-6 (4.8) 

— 


1 (4.0) 

— 

0 (3.9) 



SCI 

-8 (4.8) 


— 

-6 (4.6) 

— 


1 (3.7) 

— 

1 (3.5) 



REA 

9 (2.1) 

O 

— 

8(2.1) O 

— 


3 (1.9) 

— 

3 (1.8) 


Schools Support Academic Success 

MAT 

8 (2.1) 

O 

— 

8(2.0) O 

— 


3 (1.8) 

— 

3 (1.8) 



SCI 

8 (2.2) 

O 

— 

8(2.2) O 

— 


3 (1.9) 

— 

3 (1.9) 



REA 

-3 (2.0) 


— 

-3 (2.0) 

— 


-2 (1.9) 

— 

-1 (1.8) 


Adequate Environment and Resources 

MAT 

-3 (2.2) 


— 

-3 (2.2) 

— 


-1 (2.1) 

— 

-1 (2.1) 



SCI 

-4 (2.2) 


— 

-4 (2.1) 

— 


-2 (1.9) 

— 

-2 (1.8) 


School Instruction 


REA 

— 


-1 (2.4) 

-2 (2.3) 

— 


— 

0 (1.7) 

-1 (1.7) 


Early Emphasis on Reading Skills 

MAT 

— 


-1 (2.5) 

-2 (2.4) 

— 


— 

-1 (1.8) 

-1 (1.8) 



SCI 

— 


-1 (2.5) 

-2 (2.3) 

— 


— 

-1 (1.8) 

-1 (1.8) 


Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 


-13 (6.1) ® 

-8 (5.5) 

— 


— 

8 (5.5) 

7 (5.3) 


MAT 

— 


-16 (6.5) ® 

-11 (5.7) 

— 


— 

3 (6.1) 

3 (5.8) 


SCI 

— 


-15 (6.0) ® 

-9 (5.5) 

— 


— 

5 (5.5) 

5 (5.4) 


Home Background Control Variables 

Students within Schools 


REA 

— 


— 

— 

18 (0.8) 

O 

18 (0.8) O 

18 (0.8) O 

18 (0.8) 

O 

Home Resources for Learning 

MAT 

— 


— 

— 

16 (0.8) 

O 

16 (0.8) O 

16 (0.8) O 

16 (0.8) 

O 


SCI 

— 


— 

— 

18 (0.8) 

O 

18 (0.8) O 

18 (0.8) O 

18 (0.8) 

O 


REA 

— 


— 

— 

13 (0.9) 

O 

13 (0.9) O 

13 (0.9) O 

13 (0.9) 

O 

Early Literacy/Numeracy Tasks 

MAT 

— 


— 

— 

13 (0.8) 

O 

13 (0.8) O 

13 (0.8) O 

13 (0.8) 

O 


SCI 

— 


— 

— 

11 (0.9) 

O 

11 (0.9) O 

11 (0.9) O 

11 (0.9) 

O 


Between Schools 

School Average of 
Home Resources for Learning 


School Average of 

Early Literacy/Numeracy Tasks 

( ) Standard errors appear in parentheses. 


REA 

— 

— 

— 

21 (2.6) O 

20(2.5) O 

22(2.6) O 

22(2.5) O 

MAT 

— 

— 

— 

19 (2.8) O 

19 (2.6) O 

20 (2.8) O 

19(2.7) O 

SCI 

— 

— 

— 

20(2.5) O 

20(2.3) O 

22(2.7) O 

21 (2.5) O 

REA 

— 

— 

— 

7 (5.2) 

6 (5.4) 

8 (5.3) 

7 (5.4) 

MAT 

— 

— 

— 

10 (5.1) 

9 (5.3) 

10 (5.3) 

9 (5.4) 

SCI 

— 

— 

— 

9 (5.2) 

8 (5.5) 

9 (5.2) 

7 (5.5) 


REA - Reading 
MAT - Mathematics 
SCI -Science 


O Coefficient significantly greaterthan zero. 
® Coefficient significantly less than zero. 


Source of Variance 

)j Percentage of Variance Explained 

ij School Explanatory Models |j 

Home 

| School Explanatory with Home Background Control Models 


School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


Reading 


Between Schools (10%) 

22 

6 

24 

64 

67 

67 

70 

Within Schools (90%) 

— 

— 

— 

25 

25 

25 

25 

Total 

2 

1 

2 

29 

29 

29 

30 

Mathematics 

Between Schools (13%) 

22 

10 

26 

59 

61 

60 

62 

Within Schools (87%) 

— 

— 

— 

26 

26 

26 

26 

Total 

3 

1 

3 

30 

30 

30 

30 

Science 

Between Schools (11%) 

23 

9 

27 

68 

70 

70 

72 

Within Schools (89%) 

— 

— 

— 

24 

24 

24 

24 

Total 

2 

1 

3 

29 

29 

29 

30 


( ) Percentage of available variance shown in parentheses. 
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Exhibit 3.26: School Effectiveness Models for Reading, Mathematics, and Science - Portugal 


TIMSS & PIRLS 
2011 



HLM Regression Coefficients 


School Explanatory Models :}■ 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 

School Environment 



REA 

2 (3.5) 


— 

-2 (3.4) 


— 


4 (3.2) 


— 


1 (2.9) 


Schools Are Safe and Orderly 

MAT 

0 (5.6) 


— 

-4 (5.4) 


— 


6 (5.3) 


— 


1 (4.9) 



SCI 

0 (5.1) 


— 

-4 (4.8) 


— 


3 (5.1) 


— 


-1 (4.7) 



REA 

9 (2.8) 

o 

— 

8 (2.6) 

o 

— 


5 (2.3) 

o 

— 


4 (2.1) 


Schools Support Academic Success 

MAT 

12 (4.3) 

o 

— 

9 (4.1) 

o 

— 


6 (3.3) 


— 


4 (3.1) 



SCI 

12 (4.2) 

o 

— 

10 (3.9) 

o 

— 


8 (3.5) 

o 

— 


6 (3.2) 



REA 

-5 (2.6) 


— 

-5 (2.4) 

® 

— 


-7 (2.4) 

® 

— 


-7 (2.2) 

® 

Adequate Environment and Resources 

MAT 

-4 (3.8) 


— 

-4 (3.6) 


— 


-8 (3.3) 

® 

— 


-8 (3.2) 

® 


SCI 

-5 (3.2) 


— 

-5 (3.0) 


— 


-7 (3.0) 

® 

— 


-7 (2.9) 

® 

School Instruction 


REA 

— 


-4 (2.4) 

-4 (2.3) 


— 


— 


-2 (2.0) 


-2 (1.8) 


Early Emphasis on Reading Skills 

MAT 

— 


-6 (3.9) 

-6 (3.8) 


— 


— 


-5 (3.5) 


-5 (3.3) 



SCI 

— 


-6 (3.5) 

-6 (3.3) 


— 


— 


-5 (3.4) 


-5 (3.3) 


Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 


21 (4.7) O 

17 (5.1) 

o 

— 


— 


19 (4.8) 

o 

17 (5.0) 

o 

MAT 

— 


25 (6.6) O 

22 (7.5) 

o 

— 


— 


23 (6.8) 

o 

21 (7.4) 

o 

SCI 

— 


26 (6.7) O 

23 (7.2) 

o 

— 


— 


24 (7.2) 

o 

22 (7.7) 

o 

Home Background Control Variables 














Students within Schools 
















REA 

— 


— 

— 


12 (1.2) 

o 

12 (1.2) 

o 

12 (1.2) 

o 

12 (1.2) 

o 

Home Resources for Learning 

MAT 

— 


— 

— 


10 (1.0) 

o 

10 (1.0) 

o 

10 (1.0) 

o 

10 (1.0) 

o 


SCI 

— 


— 

— 


11 (i.i) 

o 

11 (1.1) 

o 

11 (1.1) 

o 

11 (1.1) 

o 


REA 

— 


— 

— 


10 (0.9) 

o 

10 (0.9) 

o 

10 (0.9) 

o 

10 (0.9) 

o 

Early Literacy/Numeracy Tasks 

MAT 

— 


— 

— 


9 (0.9) 

o 

9 (0.9) 

o 

9 (0.9) 

o 

9 (0.9) 

o 


SCI 

— 


— 

— 


8 (1.0) 

o 

8 (1.1) 

o 

8 (1.0) 

o 

8 (1.1) 

o 

Between Schools 

School Average of 
Home Resources for Learning 

REA 

— 


— 

— 


15 (3.8) 

o 

14 (3.6) 

o 

13 (3.2) 

o 

13 (3.0) 

o 

MAT 

— 


— 

— 


17 (5.9) 

o 

16 (5.6) 

o 

15 (5.0) 

o 

15 (4.8) 

o 

SCI 

— 


— 

— 


14 (5.7) 

o 

12 (5.3) 

o 

13 (4.7) 

o 

12 (4.5) 

o 

School Average of 

Early Literacy/Numeracy Tasks 

REA 

— 


— 

— 


5 (4.6) 


3 (5.0) 


3 (4.6) 


2 (4.6) 


MAT 

— 


— 

— 


-13 (8.8) 


-14 (8.7) 


-15 (8.7) 


-16 (8.4) 


SCI 

— 


— 

— 


-7 (6.3) 


-7 (6.5) 


-9 (6.5) 


-9 (6.2) 


( ) Standard errors appear in parentheses. 

REA - 

Reading 

O Coefficient significantly greater than zero. 











MAT- 

Mathematics 

® Coefficient significantly less than zero. 











SCI - 

Science 















Percentage of Variance Explained 


School Explanatory Models H ome School Explanatory with Home Background Control Models 

School School School Environment Background School School School Environment 

Environment Instruction and Instruction Control Model Environment Instruction and Instruction 


Reading 

Between Schools (18%) 

26 

26 

40 

28 

44 

50 

57 

Within Schools (82%) 

— 

— 

— 

15 

15 

15 

15 

Total 

4 

5 

7 

17 

20 

21 

22 

Mathematics 

Between Schools (38%) 

15 

21 

29 

15 

24 

33 

37 

Within Schools (62%) 

— 

— 

— 

15 

15 

15 

15 

Total 

6 

8 

11 

15 

18 

22 

23 

Science 

Between Schools (34%) 

19 

23 

33 

11 

23 

31 

37 

Within Schools (66%) 

— 

— 

— 

14 

14 

14 

14 

Total 

6 

8 

11 

13 

17 

20 

22 


( ) Percentage of available variance shown in parentheses. 
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Exhibit 3.27: School Effectiveness Models for Reading, Mathematics, and Science - Qatar ^ ^ 2011 



HLM Regression Coefficients 


School Explanatory Models 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 

School Environment 



REA 

25 (6.6) 

o 

— 

18 (6.4) 

o 

— 

19 (4.4) © 

— 

17 (4.5) 

o 

Schools Are Safe and Orderly 

MAT 

18 (6.8) 

o 

— 

12 (6.7) 


— 

12 (4.5) O 

— 

11 (4.6) 

o 


SCI 

26 (7.9) 

o 

— 

20 (7.9) 

o 

— 

19 (6.1) © 

— 

19 (6.1) 

o 


REA 

12 (4.2) 

o 

— 

10 (4.0) 

o 

— 

5 (3.3) 

— 

4 (3.2) 


Schools Support Academic Success 

MAT 

10 (4.6) 

o 

— 

9 (4.5) 

o 

— 

3 (3.5) 

— 

2 (3.5) 



SCI 

12 (5.1) 

o 

— 

10 (5.1) 


— 

4 (4.3) 

— 

3 (4.3) 



REA 

8 (2.7) 

o 

— 

8 (2.5) 

o 

— 

4(1.7) O 

— 

4 (1.6) 

o 

Adequate Environment and Resources 

MAT 

11 (3.0) 

o 

— 

10 (2.9) 

o 

— 

5 (2.0) O 

— 

5 (1.9) 

o 


SCI 

8 (3.2) 

o 

— 

8 (3.1) 

o 

— 

4 (2.4) 

— 

4 (2.3) 


School Instruction 


REA 

— 


7(2.9) o 

3 (2.2) 


— 

— 

1 (2.1) 

-1 (1.8) 


Early Emphasis on Reading Skills 

MAT 

— 


7(3.1) o 

4 (2.4) 


— 

— 

0 (2.2) 

-1 (2.0) 



SCI 

— 


5 (3.6) 

2 (2.9) 


— 

— 

-1 (3.1) 

-3 (2.7) 


Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 


47 (8.5) O 

30 (7.8) 

o 

— 

— 

29(7.5) O 

19 (6.8) 

o 

MAT 

— 


36 (8.7) O 

22 (8.2) 

o 

— 

— 

20(7.5) O 

12 (7.2) 


SCI 

— 


49 (10.9) O 

31 (9.7) 

o 

— 

— 

27 (10.3) O 

16 (9.6) 


Home Background Control Variables 











Students within Schools 













REA 

— 


— 

— 


14 (1.4) o 

14 (1.4) O 

14 (1.4) O 

14 (1.4) 

o 

Home Resources for Learning 

MAT 

— 


— 

— 


11 (1.4) o 

11 (1.4) O 

11 (1.4) O 

11 (1.4) 

o 


SCI 

— 


— 

— 


15 (1.3) o 

15(1.3) O 

15 (1.3) O 

15 (1.3) 

o 


REA 

— 


— 

— 


16 (1.3) o 

16 (1.3) O 

16 (1.3) O 

16 (1.3) 

o 

Early Literacy/Numeracy Tasks 

MAT 

— 


— 

— 


13 (1.3) o 

13(1.3) O 

13 (1.3) O 

13 (1.3) 

o 


SCI 

— 


— 

— 


18 (1.4) O 

18 (1.4) O 

18 (1.4) O 

18 (1.4) 

o 

Between Schools 

School Average of 
Home Resources for Learning 

REA 

— 


— 

— 


69 (5.9) O 

57(5.2) O 

64 (6.1) O 

56 (5.4) 

o 

MAT 

— 


— 

— 


71 (5.1) O 

61 (5.4) O 

68 (5.5) O 

61 (5.5) 

o 

SCI 

— 


— 

— 


70 (6.8) O 

58 (6.7) O 

67(7.5) O 

59 (7.2) 

o 

School Average of 

Early Literacy/Numeracy Tasks 

REA 

— 


— 

— 


31 (11.2) O 

28 (9.5) O 

16 (10.2) 

18 (9.5) 


MAT 

— 


— 

— 


18 (11.0) 

17 (9.8) 

8 (10.1) 

11 (9.7) 


SCI 

— 


— 

— 


48 (14.2) O 

45 (12.2) O 

33 (12.7) O 

36 (12.0) O 


() Standard errors appear in parentheses. REA -Reading © Coefficient significantlygreaterthan zero. 


MAT 

-Mathematics ® Coefficient significantly less than zero. 





SCI 

- Science 









Percentage of Variance Explained 

Source of Variance 


School Explanatory Models 


Home 


School Explanatory with Home Background Control Models 



School 

School 

School Environment 


Background 


School 

School 

School Environment 



Environment 

Instruction 

and Instruction 


Control Model 


Environment 

Instruction 

and Instruction 

Reading 








Between Schools (38%) 

37 

22 

44 

60 

71 

65 

73 

Within Schools (62%) 

— 

— 

— 

11 

11 

11 

11 

Total 

14 

8 

17 

29 

34 

31 

35 

Mathematics 








Between Schools (45%) 

30 

15 

35 

61 

68 

63 

69 

Within Schools (55%) 

— 

— 

— 

9 

9 

9 

9 

Total 

14 

7 

16 

32 

36 

34 

36 

Science 








Between Schools (42%) 

29 

16 

34 

50 

59 

53 

60 

Within Schools (58%) 

— 

— 

— 

12 

12 

12 

12 

Total 

12 

7 

14 

28 

32 

29 

32 


( ) Percentage of available variance shown in parentheses. 
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Exhibit 3.28: School Effectiveness Models for Reading, Mathematics, and Science - Romania 


TIMSS & PIRLS 
2011 


4 th 

Grade 



| HLM Regression Coefficients 

Variables 

| School Explanatory Models ji 

Home 

| School Explanatory with Home Background Control Models 


School 

School 

School Environment 

Background 

School 

School 

School Environment 


Environment 

Instruction 

and Instruction 

Control Model 

Environment 

Instruction 

and Instruction 


School Explanatory Variables 


School Environment 



REA 

11 (7.5) 

— 


6 (8.0) 

— 

9 (7.4) 

— 

7 (7.5) 

Schools Are Safe and Orderly 

MAT 

11 (9.9) 

— 


10 (10.7) 

— 

11 (10.0) 

— 

12 (10.3) 


SCI 

13 (9.5) 

— 


8 (10.2) 

— 

10 (9.1) 

— 

9 (9.1) 


REA 

10 (4.8) 

— 


11 (5.2) O 

— 

2 (5.2) 

— 

2 (5.3) 

Schools Support Academic Success 

MAT 

12 (6.4) 

— 


13 (6.7) 

— 

8 (7.2) 

— 

7 (6.9) 


SCI 

12(5.8) O 

— 


13 (6.2) O 

— 

6 (6.8) 

— 

7 (6.6) 


REA 

1 (4.3) 

— 


1 (4.1) 

— 

-3 (4.0) 

— 

-3 (4.1) 

Adequate Environment and Resources 

MAT 

1 (5.0) 

— 


1 (5.0) 

— 

-3 (4.7) 

— 

-3 (4.8) 


SCI 

1 (4.8) 

— 


1 (4.6) 

— 

-3 (4.5) 

— 

-3 (4.6) 

School Instruction 


REA 

— 

3 (6.3) 


5 (6.0) 

— 

— 

-2 (4.9) 

-1 (5.1) 

Early Emphasis on Reading Skills 

MAT 

— 

0 (7.2) 


2 (6.9) 

— 

— 

-4 (6.0) 

-2 (6.0) 


SCI 

— 

3 (7.0) 


5 (6.6) 

— 

— 

0 (5.5) 

1 (5.2) 

Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 

19 (7.8) 

o 

15 (8.4) 

— 

— 

8 (7.7) 

4 (7.4) 

MAT 

— 

12 (9.8) 


6 (10.2) 

— 

— 

2 (11.0) 

-4 (9.9) 

SCI 

— 

19 (9.7) 

o 

14 (10.2) 

— 

— 

8 (9.5) 

4 (8.6) 


Home Background Control Variables 

Students within Schools 



REA 

— 

— 

— 

16 (1.6) 

o 

16 (1.6) 

o 

16 (1.5) 

o 

16 (1.6) 

o 

Home Resources for Learning 

MAT 

— 

— 

— 

14 (2.1) 

o 

14 (2.1) 

o 

14 (2.1) 

o 

14 (2.1) 

o 


SCI 

— 

— 

— 

16 (1.8) 

o 

16 (1.8) 

o 

16 (1.8) 

o 

16 (1.8) 

o 


REA 

— 

— 

— 

9 (1.5) 

o 

9 (1.5) 

o 

9 (1.5) 

o 

9 (1.5) 

o 

Early Literacy/Numeracy Tasks 

MAT 

— 

— 

— 

10 (2.5) 

o 

10 (2.4) 

o 

10 (2.5) 

o 

10 (2.4) 

o 


SCI 

— 

— 

— 

10 (1.7) 

o 

10 (1.7) 

o 

10 (1.7) 

o 

10 (1.7) 

o 


Between Schools 


School Average of 
Home Resources for Learning 

REA 

— 

— 

— 

25 (4.8) O 

25 (4.9) O 

24 (4.8) O 

25 (4.7) 

o 

MAT 

— 

— 

— 

21 (6.6) O 

19(7.1) O 

21 (6.7) O 

20 (6.7) 

o 

SCI 

— 

— 

— 

23(5.7) O 

21 (6.3) O 

21 (5.5) O 

20 (5.9) 

o 

School Average of 

Early Literacy/Numeracy Tasks 

REA 

— 

— 

— 

3 (7.0) 

1 (7.1) 

4 (7.2) 

2 (7.4) 


MAT 

— 

— 

— 

-1 (10.2) 

-3 (10.0) 

1 (9.8) 

-2 (9.9) 


SCI 

— 

— 

— 

-1 (9.3) 

-3 (9.0) 

0 (9.1) 

-3 (9.0) 



() Standard errors appear in parentheses. REA -Reading O Coefficient significantly greater than zero. 

MAT - Mathematics ® Coefficient significantly less than zero. 

SCI -Science 



j Percentage of Variance Explained 

Source of Variance 

| School Explanatory Models ( 

Home 

| School Explanatory with Home Background Control Models 


School 

School 

School Environment 

Background 

School 

School 

School Environment 


Environment 

Instruction 

and Instruction 

Control Model 

Environment 

Instruction 

and Instruction 


Reading 


Between Schools (35%) 

14 

10 

20 

41 

45 

43 

46 

Within Schools (65%) 

— 

— 

— 

21 

21 

21 

21 

Total 

5 

3 

7 

28 

30 

29 

30 

Mathematics 

Between Schools (38%) 

14 

3 

16 

13 

20 

14 

20 

Within Schools (62%) 

— 

— 

— 

24 

24 

24 

24 

Total 

6 

1 

6 

19 

22 

20 

22 

Science 

Between Schools (37%) 

15 

8 

20 

23 

30 

25 

30 

Within Schools (63%) 

— 

— 

— 

23 

23 

23 

23 

Total 

6 

3 

7 

23 

25 

24 

25 


( ) Percentage of available variance shown in parentheses. 
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Exhibit 3.29: School Effectiveness Models for Reading, Mathematics, and Science 
Russian Federation 


HLM Regression Coefficients 


TIMSS &PIRLSI 
201l! 



School Explanatory Models 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 

School Environment 


REA 


3 (4.2) 


4 (4.5) 


6 (3.7) 


Between Schools 

School Average of 
Home Resources for Learning 


School Average of 

Early Literacy/Numeracy Tasks 

( ) Standard errors appear in parentheses. 


REA - Reading 
MAT - Mathematics 
SCI -Science 


O Coefficient significantly greaterthan zero. 
® Coefficient significantly less than zero. 


5 (4.1) 


Schools Are Safe and Orderly 

MAT 

6 (5.3) 

— 

8 (5.5) 

— 


8 (4.6) 


— 


8 (4.6) 



SCI 

5 (5.2) 

— 

5 (5.4) 

— 


7 (4.6) 


— 


6 (4.7) 



REA 

3 (3.2) 

— 

3 (3.3) 

— 


-2 (2.9) 


— 


-3 (2.8) 


Schools Support Academic Success 

MAT 

0 (3.3) 

— 

0 (3.4) 

— 


-5 (2.9) 


— 


-5 (2.9) 



SCI 

2 (3.6) 

— 

1 (3.6) 

— 


-4 (3.2) 


— 


-4 (3.2) 



REA 

3 (2.2) 

— 

3 (2.3) 

— 


1 (2.1) 


— 


2 (2.1) 


Adequate Environment and Resources 

MAT 

1 (2.3) 

— 

2 (2.4) 

— 


0 (2.3) 


— 


1 (2.3) 



SCI 

3 (2.5) 

— 

3 (2.5) 

— 


2 (2.4) 


— 


2 (2.5) 


School Instruction 


REA 

— 

-1 (1.7) 

-2 (1.8) 

— 


— 


-2 (1.6) 


-2 (1.6) 


Early Emphasis on Reading Skills 

MAT 

— 

-2 (2.4) 

-4 (2.4) 

— 


— 


-2 (2.2) 


-3 (2.2) 



SCI 

— 

-1 (2.3) 

-2 (2.4) 

— 


— 


-1 (2.1) 


-2 (2.2) 


Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 

1 (3.4) 

0 (3.6) 

— 


— 


5 (3.0) 


4 (3.2) 


MAT 

— 

1 (4.2) 

-1 (4.4) 

— 


— 


4 (3.8) 


2 (4.0) 


SCI 

— 

2 (4.4) 

1 (4.7) 

— 


— 


5 (3.9) 


3 (4.1) 


Home Background Control Variables 

Students within Schools 


REA 

— 

— 

— 

10 (1.2) 

O 

10 (1.2) 

O 

10 (1.2) 

O 

10 (1.2) 

O 

Home Resources for Learning 

MAT 

— 

— 

— 

8 (1.0) 

O 

8 (1.0) 

O 

8 (1.0) 

O 

8 (1.0) 

O 


SCI 

— 

— 

— 

8 (1.0) 

O 

8 (1.0) 

O 

8 (1.0) 

O 

8 (1.0) 

O 


REA 

— 

— 

— 

11 (0.8) 

O 

11 (0.8) 

O 

11 (0.8) 

O 

11 (0.8) 

O 

Early Literacy/Numeracy Tasks 

MAT 

— 

— 

— 

10 (0.9) 

O 

10 (0.9) 

O 

10 (0.9) 

O 

10 (0.9) 

O 


SCI 

— 

— 

— 

10 (0.9) 

O 

10 (0.9) 

O 

10 (0.9) 

O 

10 (0.9) 

O 


REA 

— 

— 

— 

24 (4.2) O 

25 (4.2) O 

24 (4.1) O 

25 (4.1) O 

MAT 

— 

— 

— 

23 (5.4) O 

25(5.2) O 

22(5.4) O 

24(5.2) O 

SCI 

— 

— 

— 

24 (5.0) O 

25 (5.0) O 

23 (5.0) O 

25 (5.0) O 

REA 

— 

— 

— 

2 (4.8) 

2 (4.6) 

4 (4.8) 

3 (4.7) 

MAT 

— 

— 

— 

4 (7.2) 

3 (6.9) 

5 (7.3) 

4 (7.0) 

SCI 

— 

— 

— 

2 (6.7) 

1 (6.4) 

3 (6.6) 

3 (6.4) 



jj Percentage of Variance Explained 

Source of Variance 

| School Explanatory Models |j 

Home 

| School Explanatory with Home Background Control Models 


School 

School 

School Environment 

Background 

School 

School 

School Environment 


Environment 

Instruction 

and Instruction 

Control Model 

Environment 

Instruction 

and Instruction 


Reading 


Between Schools (25%) 

4 

0 

5 

40 

42 

42 

43 

Within Schools (75%) 

— 

— 

— 

16 

16 

16 

16 

Total 

1 

0 

1 

22 

22 

22 

22 

Mathematics 

Between Schools (34%) 

2 

1 

4 

20 

22 

21 

24 

Within Schools (66%) 

— 

— 

— 

12 

12 

12 

12 

Total 

1 

0 

1 

15 

16 

16 

16 

Science 

Between Schools (33%) 

3 

0 

4 

25 

27 

26 

28 

Within Schools (67%) 

— 

— 

— 

13 

13 

13 

13 

Total 

1 

0 

1 

16 

17 

17 

18 


( ) Percentage of available variance shown in parentheses. 
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Exhibit 3.30: School Effectiveness Models for Reading, Mathematics, and Science - 
Saudi Arabia 


HLM Regression Coefficients 


TIMSS &PIRLSI 
201ll 



School Explanatory Models | 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 

School Environment 



REA 

5 (4.1) 


— 

-1 (3.7) 


— 


5 (3.8) 


— 


0 (3.6) 

Schools Are Safe and Orderly 

MAT 

2 (5.1) 


— 

-1 (5.2) 


— 


1 (5.1) 


— 


0 (5.3) 


SCI 

4 (4.8) 


— 

-1 (4.6) 


— 


5 (4.4) 


— 


1 (4.5) 


REA 

18 (3.2) 

O 

— 

15 (3.2) 

O 

— 


14 (3.4) 

O 

— 


12(3.4) O 

Schools Support Academic Success 

MAT 

13 (4.7) 

O 

— 

12 (4.7) 

O 

— 


10 (4.6) 

O 

— 


9 (4.7) 


SCI 

18 (4.1) 

O 

— 

15 (4.1) 

O 

— 


13 (3.9) 

O 

— 


12 (3.9) O 


REA 

5 (4.5) 


— 

5 (3.8) 


— 


3 (4.1) 


— 


4 (3.7) 

Adequate Environment and Resources 

MAT 

0 (5.6) 


— 

0 (5.5) 


— 


-1 (5.8) 


— 


-1 (5.7) 


SCI 

4 (5.4) 


— 

3 (5.1) 


— 


1 (5.4) 


— 


2 (5.2) 

School Instruction 















REA 

— 


3 (2.8) 

1 (2.4) 


— 


— 


0 (2.6) 


0 (2.4) 

Early Emphasis on Reading Skills 

MAT 

— 


1 (3.6) 

0 (3.5) 


— 


— 


-1 (3.4) 


-1 (3.3) 


SCI 

— 


3 (3.3) 

1 (3.0) 


— 


— 


-1 (3.1) 


-1 (2.8) 

Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 


41 (7.4) O 

31 (7.4) 

O 

— 


— 


33 (6.4) 

O 

26 (6.8) O 

MAT 

— 


21 (7.6) O 

14 (7.6) 


— 


— 


14 (7.3) 


10 (7.9) 

SCI 

— 


38 (7.1) O 

28 (7.0) 

O 

— 


— 


30 (6.9) 

O 

23 (7.5) O 

Home Background Control Variables 













Students within Schools 















REA 

— 


— 

— 


5 (1.1) 

O 

5 (1.1) 

O 

5 (1.1) 

O 

5(1.1) O 

Home Resources for Learning 

MAT 

— 


— 

— 


5 (1.4) 

O 

5 (1.4) 

O 

5 (1.4) 

O 

5(1.4) O 


SCI 

— 


— 

— 


8 (1.4) 

O 

8 (1.4) 

O 

8 (1.4) 

O 

8(1.4) O 


REA 

— 


— 

— 


11 (1.1) 

O 

11 (1.1) 

O 

11 (1.1) 

O 

11 (1.1) O 

Early Literacy/Numeracy Tasks 

MAT 

— 


— 

— 


10 (1.1) 

O 

10 (1.1) 

O 

10 (1.1) 

O 

10 (1.1) 0 


SCI 

— 


— 

— 


10 (1.4) 

O 

10 (1.4) 

O 

10 (1.3) 

O 

10 (1.3) O 

Between Schools 














School Average of 
Home Resources for Learning 

REA 

— 


— 

— 


18 (5.6) 

O 

9 (6.0) 


16 (5.1) 

O 

9 (5.7) 

MAT 

— 


— 

— 


11 (6.7) 


6 (7.2) 


10 (6.8) 


6 (7.2) 

SCI 

— 


— 

— 


19 (6.4) 

O 

10 (6.8) 


17 (6.2) 

O 

11 (6.6) 

School Average of 

Early Literacy/Numeracy Tasks 

REA 

— 


— 

— 


18 (6.4) 

O 

16 (6.0) 

O 

9 (5.3) 


9 (5.2) 

MAT 

— 


— 

— 


15 (6.9) 

O 

13 (6.6) 


11 (6.9) 


11 (6.9) 

SCI 

— 


— 

— 


20 (6.3) 

O 

17 (5.8) 

O 

11 (5.7) 

O 

11 (5.6) 

( ) Standard errors appear in parentheses. 

REA 

- Reading 

O Coefficient significantly greater than zero. 










MAT 

- Mathematics 

® Coefficient significantly less than zero. 










SCI 

-Science 















Percentage of Variance Explained 

Source of Variance 



School Explanatory Models 

[I Home 



School Explanatory with Home Background Control Models 




School 


School 

School Environment 

!| Background 


School 


School 


School Environment 




Environment 

Instruction 

and Instruction 

r Control Model 


Environment 

Instruction 

and Instruction 

Reading 














Between Schools (36%) 


34 


31 

47 


27 


44 


41 


51 

Within Schools (64%) 


— 


— 

— 


6 


6 


6 


6 

Total 


12 


11 

17 


13 


20 


19 


22 

Mathematics 














Between Schools (37%) 


13 


7 

15 


9 


17 


12 


17 

Within Schools (63%) 


— 


— 

— 


6 


6 


6 


6 

Total 


5 


3 

6 


7 


10 


8 


10 


Science 


Between Schools (37%) 

26 

21 

35 

19 

33 

28 

38 

Within Schools (63%) 

— 

— 

— 

8 

7 

8 

8 

Total 

10 

8 

13 

12 

17 

15 

19 


( ) Percentage of available variance shown in parentheses. 
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Exhibit 3.31: School Effectiveness Models for Reading, Mathematics, and Science - Singapore 1 201 ll 



HLM Regression Coefficients 


School Explanatory Models 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 

School Environment 



REA 

7 (4.8) 

— 

8 (4.7) 

— 

1 (1-9) 

— 

2 (1.9) 

Schools Are Safe and Orderly 

MAT 

6 (4.4) 

— 

6 (4.3) 

— 

1 (2.2) 

— 

1 (2.2) 


SCI 

6 (4.8) 

— 

6 (4.7) 

— 

0 (2.1) 

— 

0 (2.1) 


REA 

13 (2.8) O 

— 

14 (2.8) O 

— 

1 (1.4) 

— 

2 (1.4) 

Schools Support Academic Success 

MAT 

12 (2.7) O 

— 

12 (2.6) O 

— 

1 (1.5) 

— 

2 (1.5) 


SCI 

14 (2.9) O 

— 

15 (2.8) O 

— 

2 (1.4) 

— 

2 (1.4) 


REA 

-5 (2.5) 

— 

-4 (2.4) 

— 

1 (1.0) 

— 

1 (1.0) 

Adequate Environment and Resources 

MAT 

-5 (2.2) ® 

— 

-4 (2.2) 

— 

0 (1.0) 

— 

0 (1.0) 


SCI 

-5 (2.4) ® 

— 

-4 (2.4) 

— 

1 (0.9) 

— 

1 (1.0) 

School Instruction 


REA 

— 

0 (2.3) 

-2 (2.0) 

— 

— 

-1 (0.8) 

-1 (0.8) 

Early Emphasis on Reading Skills 

MAT 

— 

0 (2.0) 

-1 (1.8) 

— 

— 

-1 (0.9) 

-1 (0.9) 


SCI 

— 

0 (2.3) 

-2 (2.0) 

— 

— 

-1 (0.8) 

-1 (0.8) 

Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 

9 (7.7) 

12 (6.9) 

— 

— 

6 (3.4) 

6 (3.4) 

MAT 

— 

10 (6.9) 

13 (6.2) O 

— 

— 

7(3.5) O 

7(3.5) O 

SCI 

— 

7 (7.6) 

10 (6.8) 

— 

— 

4 (3.4) 

5 (3.4) 


Home Background Control Variables 

Students within Schools 



REA 

— 

— 

— 

15 (0.7) 

o 

15 (0.7) 

o 

15 (0.7) 

o 

15 (0.7) 

o 

Home Resources for Learning 

MAT 

— 

— 

— 

12 (0.8) 

o 

12 (0.8) 

o 

12 (0.8) 

o 

12 (0.8) 

o 


SCI 

— 

— 

— 

17 (0.8) 

o 

17 (0.8) 

o 

17 (0.8) 

o 

17 (0.8) 

o 


REA 

— 

— 

— 

16 (1.1) 

o 

16 (1.1) 

o 

16 (1.1) 

o 

16 (1.1) 

o 

Early Literacy/Numeracy Tasks 

MAT 

— 

— 

— 

15 (1.1) 

o 

15 (1.1) 

o 

15 (1.1) 

o 

15 (1.1) 

o 


SCI 

— 

— 

— 

16 (1.1) 

o 

16 (1.1) 

o 

16 (1.1) 

o 

16 (1.1) 

o 

Between Schools 

School Average of 
Home Resources for Learning 

REA 

— 

— 

— 

29 (2.6) 

o 

27 (3.1) 

o 

30 (2.7) 

o 

28 (3.1) 

o 

MAT 

— 

— 

— 

25 (2.6) 

o 

23 (3.2) 

o 

26 (2.6) 

o 

23 (3.2) 

o 

SCI 

— 

— 

— 

32 (2.5) 

o 

30 (3.0) 

o 

32 (2.5) 

o 

30 (3.0) 

o 

School Average of 

Early Literacy/Numeracy Tasks 

REA 

— 

— 

— 

51 (4.4) 

o 

52 (4.5) 

o 

50 (4.5) 

o 

51 (4.6) 

o 

MAT 

— 

— 

— 

48 (4.4) 

o 

49 (4.6) 

o 

46 (4.5) 

o 

48 (4.6) 

o 

SCI 

— 

— 

— 

47 (4.3) 

o 

48 (4.5) 

o 

46 (4.4) 

o 

47 (4.6) 

o 


() Standard errors appear in parentheses. REA -Reading © Coefficient significantlygreaterthan zero. 


MAT 

SCI 

-Mathematics ® Coefficient significantly less than zero. 
- Science 







Percentage of Variance Explained 

Source of Variance 


School Explanatory Models 


Home 


School Explanatory with Home Background Control Models 



School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


Background 
Control Model 


School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Reading 








Between Schools (25%) 

23 

1 

25 

88 

88 

88 

89 

Within Schools (75%) 

— 

— 

— 

22 

22 

22 

22 

Total 

6 

0 

6 

38 

38 

38 

38 

Mathematics 








Between Schools (25%) 

22 

1 

24 

84 

84 

84 

85 

Within Schools (75%) 

— 

— 

— 

21 

21 

21 

21 

Total 

5 

0 

6 

37 

37 

37 

37 

Science 








Between Schools (25%) 

24 

1 

25 

88 

88 

88 

89 

Within Schools (75%) 

— 

— 

— 

24 

24 

24 

24 

Total 

6 

0 

6 

40 

41 

41 

41 


( ) Percentage of available variance shown in parentheses. 
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Exhibit 3.32: School Effectiveness Models for Reading, Mathematics, and Science - 
Slovak Republic 


HLM Regression Coefficients 


TIMSS &PIRLSI 
201ll 



School Explanatory Models :}■ 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 


School Environment 



REA 

8 (3.8) 

O 

— 

8 (3.7) 

O 

— 

8 (4.0) 


— 

7 (3.7) 


Schools Are Safe and Orderly 

MAT 

14 (5.0) 

O 

— 

13 (4.9) 

O 

— 

13 (5.2) 

O 

— 

12 (4.9) 

O 


SCI 

12 (4.6) 

O 

— 

11 (4.6) 

O 

— 

12 (4.9) 

O 

— 

10 (4.6) 

O 


REA 

9 (3.1) 

O 

— 

9 (3.2) 

O 

— 

3 (2.6) 


— 

3 (2.7) 


Schools Support Academic Success 

MAT 

10 (4.3) 

O 

— 

11 (4.3) 

O 

— 

5 (3.6) 


— 

5 (3.4) 



SCI 

9 (4.0) 

O 

— 

10 (4.0) 

O 

— 

4 (3.3) 


— 

4 (3.2) 



REA 

-2 (2.8) 


— 

-2 (2.8) 


— 

-2 (2.6) 


— 

-2 (2.6) 


Adequate Environment and Resources 

MAT 

-4 (3.2) 


— 

-4 (3.1) 


— 

-4 (3.2) 


— 

-4 (3.1) 



SCI 

-4 (3.4) 


— 

-4 (3.3) 


— 

-3 (3.2) 


— 

-4 (3.1) 


School Instruction 


REA 

— 


1 (2.2) 

1 (1-9) 


— 

— 


0 (1.8) 

0 (1.6) 


Early Emphasis on Reading Skills 

MAT 

— 


2 (2.7) 

1 (2.4) 


— 

— 


2 (2.4) 

2 (2.2) 



SCI 

— 


1 (2.6) 

1 (2.3) 


— 

— 


1 (2.2) 

1 (2.1) 


Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 


0 (4.6) 

0 (4.1) 


— 

— 


3 (5.1) 

2 (4.7) 


MAT 

— 


4 (6.2) 

5 (5.6) 


— 

— 


9 (6.5) 

8 (5.9) 


SCI 

— 


3 (5.2) 

4 (4.7) 


— 

— 


7 (5.8) 

6 (5.2) 



Home Background Control Variables 

Students within Schools 



REA 

— 

— 

— 

16 (0.8) 

O 

16 (0.8) 

O 

16 (0.8) 

O 

16 (0.8) 

O 

Home Resources for Learning 

MAT 

— 

— 

— 

17 (0.9) 

O 

17 (0.9) 

O 

17 (0.9) 

O 

17 (0.9) 

O 


SCI 

— 

— 

— 

17 (1.0) 

O 

17 (1.0) 

O 

17 (1.0) 

O 

17 (1.0) 

O 


REA 

— 

— 

— 

8 (0.7) 

O 

8 (0.7) 

O 

8 (0.7) 

O 

8 (0.7) 

O 

Early Literacy/Numeracy Tasks 

MAT 

— 

— 

— 

8 (0.9) 

O 

8 (0.9) 

O 

8 (0.9) 

O 

8 (0.9) 

O 


SCI 

— 

— 

— 

7 (0.8) 

O 

7 (0.8) 

O 

7 (0.8) 

O 

7 (0.8) 

O 


Between Schools 


School Average of 
Home Resources for Learning 

REA 

— 

— 

— 

17 (4.7) O 

15 (4.6) O 

18 (4.6) O 

15 (4.5) 

O 

MAT 

— 

— 

— 

19(5.7) O 

15 (5.4) O 

20(5.5) O 

16 (5.2) 

O 

SCI 

— 

— 

— 

18(5.2) O 

15 (5.0) O 

19 (4.9) O 

16 (4.8) 

O 

School Average of 

Early Literacy/Numeracy Tasks 

REA 

— 

— 

— 

-6 (7.5) 

-5 (6.7) 

-7 (7.2) 

-6 (6.6) 


MAT 

— 

— 

— 

-13 (9.4) 

-12 (8.1) 

-16 (8.8) 

-14 (7.8) 


SCI 

— 

— 

— 

-12 (8.7) 

-11 (7.5) 

-14 (8.2) 

-12 (7.2) 



() Standard errors appear in parentheses. REA -Reading O Coefficient significantly greater than zero. 

MAT - Mathematics ® Coefficient significantly less than zero. 

SCI -Science 



| Percentage of Variance Explained 

Source of Variance 

| School Explanatory Models ( 

Home 

| School Explanatory with Home Background Control Models 


School 

School 

School Environment 

Background 

School 

School 

School Environment 


Environment 

Instruction 

and Instruction 

Control Model 

Environment 

Instruction 

and Instruction 


Reading 


Between Schools (18%) 

19 

0 

20 

29 

35 

30 

35 

Within Schools (82%) 

— 

— 

— 

18 

18 

18 

18 

Total 

4 

0 

4 

20 

21 

20 

21 

Mathematics 

Between Schools (27%) 

20 

1 

21 

16 

26 

19 

29 

Within Schools (73%) 

— 

— 

— 

20 

20 

20 

20 

Total 

5 

0 

6 

19 

21 

19 

22 

Science 

Between Schools (26%) 

17 

1 

18 

17 

25 

20 

27 

Within Schools (74%) 


— 

— 

19 

19 

19 

19 

Total 

■ 

0 

5 

18 

20 

19 

21 


( ) Percentage of available variance shown in parentheses. 
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Exhibit 3.33: School Effectiveness Models for Reading, Mathematics, and Science - Slovenia 2011 



HLM Regression Coefficients 


School Explanatory Models 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 


School Environment 



REA 

1 (2.8) 

— 

2 (3.0) 

— 

1 (2.6) 

— 

2 (2.5) 

Schools Are Safe and Orderly 

MAT 

1 (2.4) 

— 

2 (2.6) 

— 

1 (2.2) 

— 

1 (2.2) 


SCI 

2 (2.9) 

— 

3 (2.9) 

— 

1 (2.6) 

— 

2 (2.5) 


REA 

3 (1.9) 

— 

4(1.8) O 

— 

0 (1.8) 

— 

0 (1.8) 

Schools Support Academic Success 

MAT 

4(1.7) O 

— 

4(1.7) O 

— 

0 (1.7) 

— 

1 (1.8) 


SCI 

3 (2.0) 

— 

3 (1.9) 

— 

-1 (1.9) 

— 

-1 (2.0) 


REA 

0 (1.5) 

— 

0 (1.5) 

— 

2 (1.2) 

— 

2 (1.2) 

Adequate Environment and Resources 

MAT 

0 (1.5) 

— 

-1 (1.6) 

— 

1 (1.2) 

— 

1 (1.3) 


SCI 

0 (1.7) 

— 

0 (1.7) 

— 

1 (1.4) 

— 

1 (1.5) 

School Instruction 


REA 

— 

-1 (1.2) 

-1 (1.3) 

— 

— 

0 (1.2) 

0 (1.2) 

Early Emphasis on Reading Skills 

MAT 

— 

-1 (1.4) 

-2 (1.4) 

— 

— 

-1 (1.3) 

-1 (1.3) 


SCI 

— 

-1 (1.6) 

-2 (1.7) 

— 

— 

-1 (1.5) 

-1 (1.5) 

Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 

-3 (3.9) 

-4 (3.8) 

— 

— 

-1 (3.7) 

-2 (3.6) 

MAT 

— 

-5 (4.7) 

-6 (4.7) 

— 

— 

-4 (4.6) 

-4 (4.6) 

SCI 

— 

-5 (4.5) 

-6 (4.4) 

— 

— 

-4 (4.5) 

-4 (4.5) 


Home Background Control Variables 

Students within Schools 



REA 

— 

— 

— 

21 (0.9) 

o 

21 (0.9) 

o 

21 (0.9) 

o 

21 (0.9) 

o 

Home Resources for Learning 

MAT 

— 

— 

— 

19 (1.2) 

o 

19 (1.2) 

o 

19 (1.2) 

o 

19 (1.2) 

o 


SCI 

— 

— 

— 

22 (1.3) 

o 

22 (1.3) 

o 

22 (1.3) 

o 

22 (1.3) 

o 


REA 

— 

— 

— 

11 (0.8) 

o 

11 (0.8) 

o 

11 (0.8) 

o 

11 (0.8) 

o 

Early Literacy/Numeracy Tasks 

MAT 

— 

— 

— 

11 (0.8) 

o 

11 (0.8) 

o 

11 (0.8) 

o 

11 (0.7) 

o 


SCI 

— 

— 

— 

9 (0.9) 

o 

9 (0.9) 

o 

9 (0.9) 

o 

9 (0.9) 

o 


Between Schools 


School Average of 
Home Resources for Learning 

School Average of 

Early Literacy/Numeracy Tasks 


REA 

— 

— 

— 

26 (2.9) O 

26(3.1) O 

26 (2.8) O 

25 (3.1) O 

MAT 

— 

— 

— 

26 (3.0) O 

25(3.2) O 

25 (2.9) O 

25(3.1) O 

SCI 

— 

— 

— 

29(3.5) O 

29 (3.8) O 

28 (3.4) O 

28 (3.7) O 

REA 

— 

— 

— 

7 (3.8) 

8 (3.9) 

7 (3.8) 

8 (3.9) 

MAT 

— 

— 

— 

8 (4.6) 

9 (4.6) 

8 (4.5) 

9 (4.6) 

SCI 

— 

— 

— 

2 (5.2) 

3 (5.2) 

2 (5.2) 

3 (5.2) 


0 Standard errors appear in parentheses. REA -Reading © Coefficient significantlygreaterthan zero. 

MAT - Mathematics ® Coefficient significantly less than zero. 

SCI -Science 



jj Percentage of Variance Explained 

Source of Variance 

| School Explanatory Models |j 

Home 

| School Explanatory with Home Background Control Models 


School 

School 

School Environment 

Background 

School 

School 

School Environment 


Environment 

Instruction 

and Instruction 

Control Model 

Environment 

Instruction 

and Instruction 


Reading 


Between Schools (5%) 

5 

1 

8 

52 

54 

52 

55 

Within Schools (95%) 

— 

— 

— 

23 

23 

23 

23 

Total 

0 

0 

0 

25 

25 

25 

25 

Mathematics 

Between Schools (8%) 

4 

3 

10 

44 

45 

46 

47 

Within Schools (92%) 

— 

— 

— 

26 

26 

26 

26 

Total 

0 

0 

1 

28 

28 

28 

28 

Science 

Between Schools (8%) 

3 

3 

8 

44 

45 

45 

46 

Within Schools (92%) 

— 

— 

— 

24 

24 

24 

24 

Total 

0 

0 

1 

26 

26 

26 

26 


( ) Percentage of available variance shown in parentheses. 
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Exhibit 3.34: School Effectiveness Models for Reading, Mathematics, and Science - Spain 


TIMSS & PIRLS 
2011 



HLM Regression Coefficients 


School Explanatory Models | 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 

School Environment 



REA 

5 (3.3) 


— 

5 (3.1) 


— 


5 (2.9) 

— 

5 (2.8) 


Schools Are Safe and Orderly 

MAT 

7 (3.5) 

o 

— 

6 (3.3) 

o 

— 


7(2.8) O 

— 

6 (2.7) 

o 


SCI 

6 (3.6) 


— 

5 (3.5) 


— 


6 (3.2) 

— 

5 (3.1) 



REA 

8 (3.0) 

o 

— 

8 (2.6) 

o 

— 


0 (2.9) 

— 

1 (2.6) 


Schools Support Academic Success 

MAT 

9 (2.7) 

o 

— 

10 (2.4) 

o 

— 


1 (2.2) 

— 

2 (2.1) 



SCI 

8 (2.9) 

o 

— 

8 (2.6) 

o 

— 


1 (2.7) 

— 

1 (2.5) 



REA 

-2 (2.0) 


— 

-2 (1.9) 


— 


0 (1.7) 

— 

0 (1.7) 


Adequate Environment and Resources 

MAT 

-3 (2.0) 


— 

-3 (1.9) 


— 


-1 (1.7) 

— 

0 (1.6) 



SCI 

-3 (2.2) 


— 

-2 (2.2) 


— 


0 (2.0) 

— 

0 (2.0) 


School Instruction 


REA 

— 


-3 (1.6) 

-4 (1.6) 

® 

— 


— 

-1 (1.3) 

-2 (1.3) 


Early Emphasis on Reading Skills 

MAT 

— 


-1 (1.5) 

-3 (1.5) 

® 

— 


— 

0 (1.2) 

-1 (1.4) 



SCI 

— 


-2 (1.7) 

-3 (1.7) 


— 


— 

-1 (1.4) 

-1 (1.5) 


Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 


9 (4.6) 

7 (3.7) 


— 


— 

6 (3.9) 

5 (3.7) 


MAT 

— 


10 (4.3) O 

8 (3.3) 

o 

— 


— 

8(3.2) O 

6 (2.8) 

o 

SCI 

— 


8 (4.7) 

5 (3.9) 


— 


— 

5 (4.0) 

4 (3.7) 


Home Background Control Variables 

Students within Schools 


REA 

— 


— 

— 


11 (1.0) 

o 

ii (i.o) o 

11 (1.0) o 

11 (1.0) 

o 

Home Resources for Learning 

MAT 

— 


— 

— 


11 (0.9) 

o 

11 (0.9) o 

11 (0.9) O 

11 (0.9) 

o 


SCI 

— 


— 

— 


13 (1.0) 

o 

13 (1.0) O 

13 (1.0) O 

13 (1.0) 

o 


REA 

— 


— 

— 


12 (1.0) 

o 

12 (1.0) O 

12 (1.0) O 

12 (1.0) 

o 

Early Literacy/Numeracy Tasks 

MAT 

— 


— 

— 


12 (1.0) 

o 

12 (1.0) O 

12 (1.0) O 

12 (1.0) 

o 


SCI 

— 


— 

— 


12 (1.1) 

o 

12 (1.1) O 

12 (1.1) O 

12 (1.1) 

o 


Between Schools 

School Average of 
Home Resources for Learning 


School Average of 

Early Literacy/Numeracy Tasks 

( ) Standard errors appear in parentheses. 


REA 

— 

— 

— 

17 (2.7) 

o 

16 (3.0) 

o 

17 (2.8) 

o 

15 (3.0) 

o 

MAT 

— 

— 

— 

21 (2.9) 

o 

19 (2.9) 

o 

20 (2.8) 

o 

18 (2.9) 

o 

SCI 

— 

— 

— 

17 (3.1) 

o 

15 (3.1) 

o 

16 (3.1) 

o 

14 (3.1) 

o 

REA 

— 

— 

— 

22 (5.3) 

o 

20 (5.7) 

o 

22 (5.2) 

o 

20 (5.5) 

o 

MAT 

— 

— 

— 

22 (4.8) 

o 

19 (4.7) 

o 

23 (4.6) 

o 

19 (4.4) 

o 

SCI 

— 

— 

— 

25 (5.2) 

o 

22 (5.8) 

o 

25 (5.1) 

o 

23 (5.7) 

o 


REA - Reading 
MAT - Mathematics 
SCI -Science 


© Coefficient significantly greater than zero. 
® Coefficient significantly less than zero. 



Percentage of Variance Explained 


School Explanatory Models H ome School Explanatory with Home Background Control Models 

School School School Environment Background School School School Environment 

Environment Instruction and Instruction Control Model Environment Instruction and Instruction 


Reading 

Between Schools (18%) 

23 

6 

29 

40 

45 

44 

48 

Within Schools (82%) 

— 

— 

— 

15 

15 

15 

15 

Total 

4 

1 

5 

20 

21 

20 

21 

Mathematics 

Between Schools (21%) 

34 

7 

39 

55 

62 

58 

64 

Within Schools (79%) 

— 

— 

— 

18 

18 

18 

18 

Total 

7 

1 

8 

26 

27 

26 

28 

Science 

Between Schools (19%) 

24 

4 

28 

42 

47 

44 

49 

Within Schools (81%) 

— 

— 

— 

18 

18 

18 

18 

Total 

5 

1 

5 

23 

24 

23 

24 


( ) Percentage of available variance shown in parentheses. 
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Exhibit 3.35: School Effectiveness Models for Reading, Mathematics, and Science - Sweden 201l! 



HLM Regression Coefficients 


School Explanatory Models 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 


School Environment 



REA 

11 (2.7) 

o 

— 

11 (2.8) 

o 

— 

2 (2.1) 

— 

2 (2.2) 

Schools Are Safe and Orderly 

MAT 

9 (2.4) 

o 

— 

9 (2.3) 

o 

— 

2 (1.9) 

— 

2 (1.9) 


SCI 

14 (3.1) 

o 

— 

14 (3.1) 

o 

— 

4 (2.4) 

— 

4 (2.5) 


REA 

2 (2.1) 


— 

2 (2.1) 


— 

-1 (1.6) 

— 

-1 (1.6) 

Schools Support Academic Success 

MAT 

5 (1.7) 

o 

— 

5 (1.7) 

o 

— 

3 (1.3) 

— 

2 (1.4) 


SCI 

3 (2.1) 


— 

3 (2.2) 


— 

0 (1.5) 

— 

0 (1.6) 


REA 

3 (2.0) 


— 

3 (2.0) 


— 

1 (1.6) 

— 

1 (1.5) 

Adequate Environment and Resources 

MAT 

1 (1.9) 


— 

1 (1.9) 


— 

0 (1.5) 

— 

0 (1.5) 


SCI 

2 (2.1) 


— 

2 (2.1) 


— 

1 (1.7) 

— 

1 (1.7) 

School Instruction 


REA 

— 


1 (1.7) 

0 (1.6) 


— 

— 

1 (1.1) 

1 (1.2) 

Early Emphasis on Reading Skills 

MAT 

— 


2 (1.4) 

0 (1.2) 


— 

— 

2 (1.0) 

1 (0.9) 


SCI 

— 


2 (1.8) 

0 (1.9) 


— 

— 

1 (1.2) 

1 (1.3) 

Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 


-3 (4.6) 

-3 (3.7) 


— 

— 

1 (3.0) 

1 (3.0) 

MAT 

— 


-5 (3.9) 

-5 (3.0) 


— 

— 

-2 (3.0) 

-2 (2.8) 

SCI 

— 


-5 (5.6) 

-5 (4.4) 


— 

— 

1 (3.6) 

0 (3.5) 


Home Background Control Variables 

Students within Schools 



REA 

— 

— 

— 

12 (0.9) 

o 

12 (0.9) 

o 

12 (0.9) 

o 

12 (0.9) 

o 

Home Resources for Learning 

MAT 

— 

— 

— 

11 (0.9) 

o 

11 (0.9) 

o 

11 (0.9) 

o 

11 (0.9) 

o 


SCI 

— 

— 

— 

15 (1.1) 

o 

15 (1.1) 

o 

15 (1.1) 

o 

15 (1.1) 

o 


REA 

— 

— 

— 

12 (0.9) 

o 

12 (0.9) 

o 

12 (0.9) 

o 

12 (0.9) 

o 

Early Literacy/Numeracy Tasks 

MAT 

— 

— 

— 

14 (1.1) 

o 

14 (1.1) 

o 

14 (1.1) 

o 

14 (1.1) 

o 


SCI 

— 

— 

— 

10 (1.1) 

o 

10 (1.1) 

o 

10 (1.1) 

o 

10 (1.1) 

o 


Between Schools 


School Average of 
Home Resources for Learning 

School Average of 

Early Literacy/Numeracy Tasks 


REA 

— 

— 

— 

29 (2.2) 

o 

28 (2.6) 

o 

29 (2.3) 

o 

28 (2.8) 

o 

MAT 

— 

— 

— 

24 (1.9) 

o 

21 (2.0) 

o 

24 (1.9) 

o 

21 (2.1) 

o 

SCI 

— 

— 

— 

35 (2.6) 

o 

32 (2.8) 

o 

35 (2.7) 

o 

32 (2.9) 

o 

REA 

— 

— 

— 

0 (5.0) 


0 (5.0) 


-1 (4.9) 


-1 (5.0) 


MAT 

— 

— 

— 

9 (4.0) 

o 

8 (4.0) 

o 

9 (4.0) 

o 

9 (4.1) 

o 

SCI 

— 

— 

— 

-7 (5.3) 


-8 (5.6) 


-8 (5.1) 


-8 (5.5) 



0 Standard errors appear in parentheses. REA -Reading © Coefficient significantlygreaterthan zero. 

MAT - Mathematics ® Coefficient significantly less than zero. 

SCI -Science 



jj Percentage of Variance Explained 

Source of Variance 

| School Explanatory Models |j 

Home 

| School Explanatory with Home Background Control Models 


School 

School 

School Environment 

Background 

School 

School 

School Environment 


Environment 

Instruction 

and Instruction 

Control Model 

Environment 

Instruction 

and Instruction 


Reading 


Between Schools (13%) 

29 

2 

29 

76 

77 

76 

77 

Within Schools (87%) 

— 

— 

— 

18 

18 

18 

18 

Total 

4 

0 

4 

25 

25 

25 

25 

Mathematics 

Between Schools (12%) 

39 

4 

41 

77 

80 

79 

81 

Within Schools (88%) 

— 

— 

— 

21 

21 

21 

21 

Total 

4 

0 

5 

28 

28 

28 

28 

Science 

Between Schools (15%) 

35 

2 

37 

78 

80 

78 

80 

Within Schools (85%) 

— 

— 

— 

19 

19 

19 

19 

Total 

5 

0 

5 

28 

28 

28 

28 


( ) Percentage of available variance shown in parentheses. 
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Exhibit 3.36: School Effectiveness Models for Reading, Mathematics, and Science - 
United Arab Emirates 


HLM Regression Coefficients 


TIMSS &PIRLSI 
201ll 



School Explanatory Models | 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 

School Environment 



REA 

17 (3.6) 

O 

— 


10 (3.4) O 

— 


12 (3.1) 

O 

— 


6 (2.9) 

O 

Schools Are Safe and Orderly 

MAT 

12 (3.5) 

O 

— 


7 (3.4) 

— 


7 (3.2) 

O 

— 


3 (3.1) 



SCI 

14 (3.7) 

O 

— 


7 (3.5) 

— 


9 (3.4) 

O 

— 


3 (3.1) 



REA 

12 (3.1) 

O 

— 


9(2.9) O 

— 


6 (2.4) 

O 

— 


5 (2.3) 

O 

Schools Support Academic Success 

MAT 

10 (2.9) 

O 

— 


8(2.7) O 

— 


6 (2.4) 

O 

— 


4 (2.3) 



SCI 

13 (3.1) 

O 

— 


11 (2.9) O 

— 


8 (2.7) 

O 

— 


7 (2.5) 

O 


REA 

7 (2.4) 

O 

— 


6(2.4) O 

— 


-1 (2.1) 


— 


-1 (2.2) 


Adequate Environment and Resources 

MAT 

6 (2.2) 

O 

— 


5 (2.3) 

— 


-1 (2.1) 


— 


-1 (2.2) 



SCI 

6 (2.3) 

O 

— 


4 (2.4) 

— 


-1 (2.2) 


— 


-1 (2.3) 


School Instruction 


REA 

— 


13 (1.6) 

O 

10 (1.6) O 

— 


— 


5 (1.4) 

O 

4 (1.3) 

O 

Early Emphasis on Reading Skills 

MAT 

— 


12 (1.5) 

O 

10 (1.5) O 

— 


— 


5 (1.5) 

O 

5 (1.4) 

O 


SCI 

— 


11 (1.6) 

O 

9(1.6) O 

— 


— 


4 (1.6) 

O 

4 (1.5) 

O 

Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 


28 (5.6) 

O 

20 (5.6) O 

— 


— 


26 (5.0) 

O 

22 (5.0) 

O 

MAT 

— 


20 (5.2) 

O 

15 (5.3) O 

— 


— 


19 (4.8) 

O 

16 (4.9) 

O 

SCI 

— 


30 (5.5) 

O 

24(5.5) O 

— 


— 


27 (5.0) 

O 

24 (5.1) 

O 

Home Background Control Variables 














Students within Schools 
















REA 

— 


— 


— 

13 (0.8) 

O 

13 (0.8) 

O 

13 (0.8) 

O 

13 (0.8) 

O 

Home Resources for Learning 

MAT 

— 


— 


— 

9 (0.7) 

O 

9 (0.7) 

O 

9 (0.7) 

O 

9 (0.7) 

O 


SCI 

— 


— 


— 

13 (1.0) 

O 

13 (1.0) 

O 

13 (1.0) 

O 

13 (1.0) 

O 


REA 

— 


— 


— 

12 (0.6) 

O 

11 (0.6) 

O 

11 (0.6) 

O 

11 (0.6) 

O 

Early Literacy/Numeracy Tasks 

MAT 

— 


— 


— 

10 (0.6) 

O 

10 (0.6) 

O 

10 (0.6) 

O 

10 (0.6) 

O 


SCI 

— 


— 


— 

13 (0.8) 

O 

13 (0.8) 

O 

13 (0.8) 

O 

13 (0.8) 

O 

Between Schools 

School Average of 
Home Resources for Learning 

REA 

— 


— 


— 

48 (2.8) 

O 

44 (2.9) 

O 

43 (2.8) 

O 

42 (3.0) 

O 

MAT 

— 


— 


— 

42 (2.6) 

O 

39 (2.9) 

O 

37 (2.7) 

O 

36 (3.0) 

O 

SCI 

— 


— 


— 

41 (2.8) 

O 

37 (3.1) 

O 

37 (3.0) 

O 

36 (3.3) 

O 

School Average of 

Early Literacy/Numeracy Tasks 

REA 

— 


— 


— 

17 (6.1) 

O 

16 (6.5) 

O 

11 (6.3) 


11 (6.7) 


MAT 

— 


— 


— 

9 (6.3) 


8 (6.6) 


6 (6.5) 


6 (6.8) 


SCI 

— 


— 


— 

22 (6.4) 

O 

20 (6.8) 

O 

15 (6.6) 

O 

13 (6.9) 


( ) Standard errors appear in parentheses. 

REA - 

Reading 

O Coefficient significantly greater than zero. 










MAT- 

Mathematics 

® Coefficient significantly less than zero. 










SCI - 

Science 















Percentage of Variance Explained 


School Explanatory Models H ome School Explanatory with Home Background Control Models 

School School School Environment Background School School School Environment 

Environment Instruction and Instruction Control Model Environment Instruction and Instruction 


Reading 

Between Schools (43%) 

21 

25 

34 

48 

54 

58 

60 

Within Schools (57%) 

— 

— 

— 

10 

10 

10 

10 

Total 

9 

10 

14 

26 

29 

30 

31 

Mathematics 

Between Schools (45%) 

16 

21 

28 

40 

44 

48 

49 

Within Schools (55%) 

— 

— 

— 

9 

9 

9 

9 

Total 

7 

9 

12 

23 

24 

26 

27 

Science 

Between Schools (41%) 

19 

23 

31 

38 

44 

49 

51 

Within Schools (59%) 

— 

— 

— 

11 

11 

11 

11 

Total 

8 

9 

13 

22 

24 

26 

27 


( ) Percentage of available variance shown in parentheses. 
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TIMSS &PIRLSI 

Exhibit 3.37: School Effectiveness Models for Reading, Mathematics, and Science - Botswana 2011 



HLM Regression Coefficients 


School Explanatory Models j; 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 

School Environment 



REA 

15 (4.8) 

o 

— 


7 (4.7) 


— 

12 (3.1) 

o 

— 

7 (3.3) 

o 

Schools Are Safe and Orderly 

MAT 

15 (3.7) 

o 

— 


8 (3.6) 

o 

— 

13 (3.0) 

o 

— 

8 (3.1) 

o 


SCI 

20 (5.7) 

o 

— 


9 (5.4) 


— 

17 (4.1) 

o 

— 

9 (4.1) 

o 


REA 

19 (2.4) 

o 

— 


15 (2.6) 

o 

— 

7 (2.1) 

o 

— 

5 (1.9) 

o 

Schools Support Academic Success 

MAT 

14 (2.1) 

o 

— 


11 (2.1) 

o 

— 

7 (2.2) 

o 

— 

5 (1.9) 

o 


SCI 

22 (2.8) 

o 

— 


17 (2.9) 

o 

— 

10 (2.9) 

o 

— 

7 (2.3) 

o 


REA 

2 (3.3) 


— 


3 (2.6) 


— 

0 (2.5) 


— 

2 (1.9) 


Adequate Environment and Resources 

MAT 

1 (2.7) 


— 


3 (1.9) 


— 

1 (2.4) 


— 

2 (1.9) 



SCI 

2 (4.1) 


— 


4 (2.9) 


— 

1 (3.5) 


— 

3 (2.5) 


School Instruction 


REA 

— 


0 (2.2) 


0 (1.9) 


— 

— 


2 (1.2) 

2 (1.2) 


Early Emphasis on Reading Skills 

MAT 

— 


0 (1.7) 


0 (1.5) 


— 

— 


1 (1.2) 

1 (1.1) 



SCI 

— 


0 (2.5) 


0 (2.2) 


— 

— 


2 (1.5) 

2 (1.5) 


Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 


55 (7.6) 

o 

37 (6.5) 

o 

— 

— 


34 (5.0) O 

28 (4.8) 

o 

MAT 

— 


47 (5.9) 

o 

32 (5.0) 

o 

— 

— 


34 (4.4) O 

27 (4.1) 

o 

SCI 

— 


75 (8.5) 

o 

54 (7.6) 

o 

— 

— 


52 (6.0) O 

44 (5.6) 

o 


Home Background Control Variables 

Students within Schools 



REA 

— 

— 

— 

6 (1.0) 

o 

6 (1.0) 

o 

6 (1.0) 

o 

6 (1.0) 

o 

Home Resources for Learning 

MAT 

— 

— 

— 

3 (0.9) 

o 

3 (0.9) 

o 

3 (0.9) 

o 

3 (0.9) 

o 


SCI 

— 

— 

— 

6 (1.2) 

o 

6 (1.2) 

o 

6 (1.2) 

o 

6 (1.2) 

o 


REA 

— 

— 

— 

10 (0.9) 

o 

10 (0.9) 

o 

10 (0.9) 

o 

10 (0.9) 

o 

Early Literacy/Numeracy Tasks 

MAT 

— 

— 

— 

9 (1.0) 

o 

9 (1.0) 

o 

9 (1.0) 

o 

9 (1.0) 

o 


SCI 

— 

— 

— 

14 (1.1) 

o 

14 (1.1) 

o 

14 (1.1) 

o 

14 (1.1) 

o 

Between Schools 

School Average of 
Home Resources for Learning 

REA 

— 

— 

— 

37 (4.8) 

o 

30 (5.2) 

o 

37 (5.0) 

o 

32 (5.2) 

o 

MAT 

— 

— 

— 

26 (4.9) 

o 

19 (5.3) 

o 

27 (4.8) 

o 

21 (5.1) 

o 

SCI 

— 

— 

— 

39 (6.4) 

o 

30 (6.9) 

o 

40 (6.4) 

o 

33 (6.6) 

o 

School Average of 

Early Literacy/Numeracy Tasks 

REA 

— 

— 

— 

9 (6.7) 


9 (5.4) 


-1 (5.6) 


1 (4.7) 


MAT 

— 

— 

— 

9 (6.4) 


8 (5.5) 


-2 (4.8) 


-1 (4.5) 


SCI 

— 

— 

— 

19 (8.8) 

o 

17 (7.3) 

o 

2 (6.4) 


4 (5.6) 



() Standard errors appear in parentheses. REA -Reading O Coefficient significantly greater than zero. 

MAT - Mathematics ® Coefficient significantly less than zero. 

SCI -Science 


Source of Variance 

! j Percentage of Variance Explained 

| School Explanatory Models j 

Home 

| School Explanatory with Home Background Control Models 


School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


Reading 


Between Schools (38%) 

44 

36 

58 

68 

77 

80 

84 

Within Schools (62%) 

— 

— 

— 

7 

7 

7 

7 

Total 

17 

13 

22 

30 

33 

35 

36 

Mathematics 

Between Schools (31%) 

46 

40 

62 

56 

70 

74 

80 

Within Schools (69%) 

— 

— 

— 

4 

4 

4 

4 

Total 

14 

12 

19 

20 

25 

26 

28 

Science 

Between Schools (35%) 

44 

43 

63 

62 

72 

79 

84 

Within Schools (65%) 

— 

— 

— 

6 

6 

6 

6 

Total 

15 

15 

22 

25 

29 

31 

33 


( ) Percentage of available variance shown in parentheses. 



TIMSS & PIRLS 

International Study Center 

Lynch School of Education, Boston College 


EFFECTIVE SCHOOLS IN READING, 
MATHEMATICS, AND SCIENCE AT THE FOURTH GRADE 

CHAPTER 3 173 


SOURCE: lEA's Trends in International Mathematics and Science Study and Progress in International Reading Literacy Study - TIMSS and PIRLS 201 1 



























Exhibit 3.38: School Effectiveness Models for Reading, Mathematics, and Science - Honduras 201 1 



HLM Regression Coefficients 


School Explanatory Models | 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 

School Environment 



REA 

7 (5.2) 

— 

6 (5.0) 

— 

8 (4.5) 

— 

6 (4.4) 

Schools Are Safe and Orderly 

MAT 

4 (5.1) 

— 

4 (5.2) 

— 

4 (5.0) 

— 

3 (5.0) 


SCI 

6 (5.3) 

— 

6 (5.3) 

— 

6 (4.9) 

— 

5 (4.9) 


REA 

-3 (3.7) 

— 

-5 (3.7) 

— 

-5 (3.2) 

— 

-5 (3.1) 

Schools Support Academic Success 

MAT 

-2 (3.9) 

— 

-3 (3.9) 

— 

-4 (3.6) 

— 

-4 (3.5) 


SCI 

-2 (4.1) 

— 

-4 (4.1) 

— 

-5 (3.7) 

— 

-5 (3.7) 


REA 

6 (4.1) 

— 

5 (3.8) 

— 

1 (3.4) 

— 

0 (3.4) 

Adequate Environment and Resources 

MAT 

8 (4.0) 

— 

7 (3.8) 

— 

3 (3.3) 

— 

2 (3.4) 


SCI 

6 (4.2) 

— 

5 (3.9) 

— 

0 (3.5) 

— 

0 (3.5) 


School instruction 



REA 

— 

9 (4.0) O 

8(3.8) O 

— 


— 


4 (3.1) 


4 (3.1) 


Early Emphasis on Reading Skills 

MAT 

— 

6 (4.3) 

5 (4.0) 

— 


— 


1 (3.3) 


1 (3.3) 



SCI 

— 

8 (4.0) 

7 (4.0) 

— 


— 


2 (3.2) 


2 (3.3) 


Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 

-6 (8.2) 

-8 (8.0) 

— 


— 


12 (7.8) 


10 (7.5) 


MAT 

— 

-4 (9.3) 

-5 (9.3) 

— 


— 


9 (8.0) 


8 (8.0) 


SCI 

— 

-9 (9.2) 

-11 (9.0) 

— 


— 


9 (8.7) 


7 (8.4) 


Home Background Control Variables 

Students within Schools 


REA 

— 

— 

— 

-2 (1.2) 


-2 (1.3) 


-2 (1.2) 


-2 (1.2) 


Home Resources for Learning 

MAT 

— 

— 

— 

-3 (1.4) 

® 

-3 (1.4) 

® 

-3 (1.4) 

® 

-3 (1.4) 

® 


SCI 

— 

— 

— 

-2 (1.3) 


-2 (1.4) 


-2 (1.3) 


-2 (1.4) 



REA 

— 

— 

— 

8 (1.8) 

o 

8 (1.8) 

o 

8 (1.8) 

o 

8 (1.8) 

o 

Early Literacy/Numeracy Tasks 

MAT 

— 

— 

— 

5 (1.6) 

o 

5 (1.6) 

o 

5 (1.6) 

o 

5 (1.6) 

o 


SCI 

— 

— 

— 

4 (1.4) 

o 

4 (1.4) 

o 

4 (1.4) 

o 

4 (1.3) 

o 

Between Schools 

School Average of 
Home Resources for Learning 

REA 

— 

— 

— 

22 (4.3) 

o 

22 (4.5) 

o 

22 (4.4) 

o 

22 (4.6) 

o 

MAT 

— 

— 

— 

17 (4.8) 

o 

16 (4.9) 

o 

18 (4.9) 

o 

17 (5.0) 

o 

SCI 

— 

— 

— 

22 (4.2) 

o 

22 (4.4) 

o 

23 (4.3) 

o 

23 (4.5) 

o 

School Average of 

Early Literacy/Numeracy Tasks 

REA 

— 

— 

— 

7 (6.5) 


8 (6.0) 


8 (6.4) 


9 (6.0) 


MAT 

— 

— 

— 

9 (7.8) 


10 (7.6) 


10 (7.8) 


11 (7.6) 


SCI 

— 

— 

— 

13 (7.2) 


13 (6.9) 


14 (7.1) 


14 (6.8) 

o 


( ) Standard errors appear in parentheses. 


REA - Reading 
MAT - Mathematics 
SCI -Science 


© Coefficient significantly greater than zero. 
® Coefficient significantly less than zero. 



Percentage of Variance Explained 


School Explanatory Models H ome School Explanatory with Home Background Control Models 

School School School Environment Background School School School Environment 

Environment Instruction and Instruction Control Model Environment Instruction and Instruction 


Reading 

Between Schools (43%) 

6 

8 

12 

32 

35 

35 

37 

Within Schools (57%) 

— 

— 

— 

7 

7 

7 

7 

Total 

3 

3 

5 

18 

19 

19 

20 

Mathematics 

Between Schools (47%) 

6 

4 

9 

23 

25 

26 

27 

Within Schools (53%) 

— 

— 

— 

6 

6 

6 

6 

Total 

3 

2 

4 

14 

15 

15 

16 

Science 

Between Schools (52%) 

4 

6 

9 

32 

33 

34 

35 

Within Schools (48%) 

— 

— 

— 

5 

5 

5 

5 

Total 

2 

3 

5 

19 

20 

20 

21 


( ) Percentage of available variance shown in parentheses. 
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Exhibit 3.39: School Effectiveness Models for Reading, Mathematics, and Science 
Quebec, Canada 


HLM Regression Coefficients 


TIMSS &PIRLSI 
201l! 



School Explanatory Models 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 

School Environment 



REA 

4 (2.4) 


— 

3 (2.3) 

— 


3 (2.0) 


— 


3 (2.0) 


Schools Are Safe and Orderly 

MAT 

4 (2.6) 


— 

4 (2.5) 

— 


3 (2.3) 


— 


3 (2.3) 



SCI 

3 (3.0) 


— 

3 (2.9) 

— 


2 (2.6) 


— 


2 (2.6) 



REA 

7 (1.9) 

O 

— 

7 (1.9) O 

— 


2 (1.7) 


— 


2 (1.6) 


Schools Support Academic Success 

MAT 

8 (1.9) 

O 

— 

8(1.9) O 

— 


4 (1.8) 

O 

— 


5 (1.8) 

O 


SCI 

8 (1.9) 

O 

— 

8(1.9) O 

— 


3 (1.7) 


— 


3 (1.7) 



REA 

1 (2.0) 


— 

2 (1.9) 

— 


1 (1.6) 


— 


1 (1.5) 


Adequate Environment and Resources 

MAT 

1 (2.2) 


— 

1 (2.2) 

— 


1 (2.0) 


— 


1 (2.0) 



SCI 

0 (1.9) 


— 

0 (1.8) 

— 


0 (1.6) 


— 


0 (1.6) 


School Instruction 


REA 

— 


0 (1.1) 

-1 (0.9) 

— 


— 


0 (0.8) 


-1 (0.7) 


Early Emphasis on Reading Skills 

MAT 

— 


-1 (1.5) 

-2 (1.3) 

— 


— 


-1 (1.2) 


-1 (1.2) 



SCI 

— 


-1 (1.3) 

-2 (1.1) 

— 


— 


-1 (0.9) 


-1 (0.9) 


Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 


11 (3.9) O 

8(3.8) O 

— 


— 


6 (3.0) 

O 

5 (3.0) 


MAT 

— 


5 (4.4) 

2 (3.9) 

— 


— 


2 (3.8) 


0 (3.6) 


SCI 

— 


10(3.7) O 

7 (3.5) 

— 


— 


6 (2.8) 

O 

5 (2.8) 


Home Background Control Variables 

Students within Schools 


REA 

— 


— 

— 

12 (0.9) 

O 

12 (0.9) 

O 

12 (0.9) 

O 

12 (0.9) 

O 

Home Resources for Learning 

MAT 

— 


— 

— 

10 (1.0) 

O 

10 (1.0) 

O 

10 (1.0) 

O 

10 (1.0) 

O 


SCI 

— 


— 

— 

12 (1.1) 

O 

12 (1.1) 

O 

12 (1.1) 

O 

12 (1.1) 

O 


REA 

— 


— 

— 

9 (0.9) 

O 

9 (0.9) 

O 

9 (0.9) 

O 

9 (0.9) 

O 

Early Literacy/Numeracy Tasks 

MAT 

— 


— 

— 

8 (1.0) 

O 

8 (1.0) 

O 

8 (1.0) 

O 

8 (1.0) 

O 


SCI 

— 


— 

— 

8 (0.7) 

O 

8 (0.7) 

O 

8 (0.7) 

O 

8 (0.7) 

O 


Between Schools 

School Average of 
Home Resources for Learning 


School Average of 

Early Literacy/Numeracy Tasks 

( ) Standard errors appear in parentheses. 


REA 

— 

— 

— 

25 (3.1) 

O 

21 (3.1) 

O 

25 (3.0) O 

21 (3.0) 

O 

MAT 

— 

— 

— 

22 (3.5) 

O 

16 (3.4) 

O 

22(3.5) O 

15 (3.3) 

O 

SCI 

— 

— 

— 

26 (3.3) 

O 

22 (3.4) 

O 

26(3.2) O 

22 (3.3) 

O 

REA 

— 

— 

— 

13 (4.8) 

O 

13 (4.8) 

O 

10 (4.9) 

10 (5.0) 

O 

MAT 

— 

— 

— 

6 (5.9) 


5 (5.7) 


5 (6.1) 

5 (5.9) 


SCI 

— 

— 

— 

7 (5.4) 


6 (5.4) 


4 (5.5) 

4 (5.5) 



REA - Reading 
MAT - Mathematics 
SCI -Science 


O Coefficient significantly greaterthan zero. 
® Coefficient significantly less than zero. 



)j Percentage of Variance Explained 

Source of Variance 

ij School Explanatory Models |j 

Home 

| School Explanatory with Home Background Control Models 


School 

School 

School Environment 

Background 

School 

School 

School Environment 


Environment 

Instruction 

and Instruction 

Control Model 

Environment 

Instruction 

and Instruction 


Reading 


Between Schools (11%) 

31 

7 

35 

58 

62 

60 

64 

Within Schools (89%) 

— 

— 

— 

12 

12 

12 

12 

Total 

3 

1 

4 

17 

17 

17 

17 

Mathematics 

Between Schools (15%) 

29 

2 

31 

32 

40 

33 

41 

Within Schools (85%) 

— 

— 

— 

10 

10 

10 

10 

Total 

4 

0 

5 

14 

15 

14 

15 

Science 

Between Schools (14%) 

29 

6 

33 

53 

57 

55 

59 

Within Schools (86%) 

— 

— 

— 

13 

13 

13 

13 

Total 

4 

1 

5 

19 

19 

19 

19 


( ) Percentage of available variance shown in parentheses. 
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Exhibit 3.40: School Effectiveness Models for Reading, Mathematics, and Science - 
Abu Dhabi, UAE 


HLM Regression Coefficients 


TIMSS &PIRLSI 
201ll 



School Explanatory Models :}■ 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 

School Environment 



REA 

14 (5.8) 

O 

— 


7 (6.0) 


— 


13 (5.0) O 

— 

8 (5.0) 

Schools Are Safe and Orderly 

MAT 

12 (6.0) 

O 

— 


7 (6.1) 


— 


11 (5.2) O 

— 

8 (5.1) 


SCI 

13 (6.2) 

O 

— 


6 (6.3) 


— 


11 (5.4) O 

— 

7 (5.3) 


REA 

15 (4.9) 

O 

— 


16 (4.7) 

O 

— 


7 (4.1) 

— 

8 (4.0) 

Schools Support Academic Success 

MAT 

13 (4.5) 

O 

— 


13 (4.3) 

O 

— 


5 (3.9) 

— 

6 (3.9) 


SCI 

16 (4.7) 

O 

— 


16 (4.5) 

O 

— 


8 (4.4) 

— 

9(4.2) O 


REA 

7 (3.5) 


— 


6 (3.8) 


— 


-1 (3.5) 

— 

-1 (3.5) 

Adequate Environment and Resources 

MAT 

6 (3.3) 


— 


5 (3.6) 


— 


-1 (3.5) 

— 

-1 (3.5) 


SCI 

6 (3.3) 


— 


6 (3.4) 


— 


-1 (3.5) 

— 

0 (3.4) 

School Instruction 


REA 

— 


7 (2.8) 

O 

6 (2.5) 

O 

— 


— 

2 (2.3) 

3 (2.1) 

Early Emphasis on Reading Skills 

MAT 

— 


7 (2.5) 

O 

6 (2.3) 

O 

— 


— 

3 (2.0) 

3 (1.9) 


SCI 

— 


7 (2.7) 

O 

6 (2.4) 

O 

— 


— 

2 (2.2) 

3 (2.0) 

Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 


26 (7.8) 

O 

21 (7.5) 

O 

— 


— 

21 (6.3) O 

17(6.1) O 

MAT 

— 


17 (7.7) 

O 

13 (7.4) 


— 


— 

12 (6.4) 

9 (6.2) 

SCI 

— 


28 (7.8) 

O 

23 (7.4) 

O 

— 


— 

21 (6.6) O 

18 (6.3) O 

Home Background Control Variables 

Students within Schools 


REA 

— 


— 


— 


11 (1.5) 

O 

11 (1.5) O 

11 (1.5) O 

11 (1.5) O 

Home Resources for Learning 

MAT 

— 


— 


— 


8 (1.4) 

O 

8(1.4) O 

8 (1.3) O 

8(1.3) O 


SCI 

— 


— 


— 


10 (1.8) 

O 

10 (1.8) O 

10 (1.8) O 

10 (1.8) O 


REA 

— 


— 


— 


12 (1.2) 

O 

12(1.2) O 

12 (1.2) O 

12(1.2) O 

Early Literacy/Numeracy Tasks 

MAT 

— 


— 


— 


11 (1.2) 

O 

11 (1.2) O 

11 (1.2) O 

11 (1.2) O 


SCI 

— 


— 


— 


14 (1.3) 

O 

14(1.3) O 

14(1.3) O 

14(1.3) O 

Between Schools 

School Average of 
Home Resources for Learning 

REA 

— 


— 


— 


44 (5.3) 

O 

40 (5.8) O 

43 (5.7) O 

39 (6.1) O 

MAT 

— 


— 


— 


40 (4.6) 

O 

37(5.5) O 

38 (4.7) O 

35 (5.6) O 

SCI 

— 


— 


— 


38 (5.3) 

O 

33 (6.2) O 

37 (5.5) O 

32(6.3) O 

School Average of 

Early Literacy/Numeracy Tasks 

REA 

— 


— 


— 


32 (9.9) 

O 

29 (10.3) O 

24 (10.4) O 

24 (10.6) O 

MAT 

— 


— 


— 


25 (9.7) 

O 

24 (10.0) O 

22 (9.9) O 

22 (10.0) O 

SCI 

— 


— 


— 


39 (9.8) 

O 

36 (10.2) O 

31 (10.1) O 

30 (10.3) O 


Of 

c 

T3 

(0 

cc 


ra 

c 

O 

ro 


() Standard errors appear in parentheses. REA -Reading O Coefficient significantly greater than zero. 

MAT - Mathematics i Coefficient significantly less than zero. 

SCI -Science 



j Percentage of Variance Explained 

Source of Variance 

| School Explanatory Models ( 

Home 

| School Explanatory with Home Background Control Models 


School 

School 

School Environment 

Background 

School 

School 

School Environment 


Environment 

Instruction 

and Instruction 

Control Model 

Environment 

Instruction 

and Instruction 


Reading 


Between Schools (40%) 

23 

14 

32 

49 

55 

55 

59 

Within Schools (60%) 

— 

— 

— 

10 

10 

10 

10 

Total 

9 

6 

13 

25 

28 

28 

30 

Mathematics 

Between Schools (42%) 

19 

11 

26 

44 

49 

48 

51 

Within Schools (58%) 

— 

— 

— 

9 

9 

9 

9 

Total 

8 

5 

11 

24 

26 

26 

27 

Science 

Between Schools (38%) 

23 

15 

32 

43 

50 

50 

55 

Within Schools (62%) 

— 

— 

— 

10 

10 

10 

10 

Total 

8 

6 

12 

22 

25 

25 

27 


( ) Percentage of available variance shown in parentheses. 
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Exhibit 3.40: School Effectiveness Models for Reading, Mathematics, and Science 
Dubai, UAE 


HLM Regression Coefficients 


TIMSS &PIRLSI 
201l! 



School Explanatory Models 

Home 

| School Explanatory with Home Background Control Models 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


School Explanatory Variables 

School Environment 



REA 

27(7.6) © 

— 

17 (6.2) O 

— 

9 (4.9) 

— 

2 (5.1) 

Schools Are Safe and Orderly 

MAT 

23 (6.7) O 

— 

14(5.5) O 

— 

7 (4.2) 

— 

1 (4.7) 


SCI 

26(7.5) O 

— 

14 (6.0) O 

— 

9 (4.8) 

— 

1 (5.2) 


REA 

8 (4.7) 

— 

4 (4.4) 

— 

6(2.6) O 

— 

6(2.7) O 

Schools Support Academic Success 

MAT 

7 (4.2) 

— 

4 (4.0) 

— 

6(2.3) O 

— 

5 (2.4) 


SCI 

8 (4.6) 

— 

5 (4.2) 

— 

7 (2.6) O 

— 

6(2.7) O 


REA 

9(3.9) O 

— 

10 (3.3) O 

— 

3 (2.5) 

— 

5 (2.7) 

Adequate Environment and Resources 

MAT 

8(3.6) O 

— 

9(3.1) O 

— 

3 (2.4) 

— 

4 (2.6) 


SCI 

8 (4.0) O 

— 

10 (3.4) O 

— 

3 (2.8) 

— 

5 (2.9) 

School Instruction 


REA 

— 

17(2.4) O 

15 (2.3) O 

— 

— 

3 (1.8) 

3 (1.8) 

Early Emphasis on Reading Skills 

MAT 

— 

15 (2.2) O 

14(2.2) O 

— 

— 

3 (1.7) 

3 (1.7) 


SCI 

— 

16 (2.4) O 

15 (2.4) O 

— 

— 

3 (1.9) 

3 (1.9) 

Students Engaged in Reading, 
Mathematics, and Science Lessons 

REA 

— 

48 (13.4) O 

35 (10.0) O 

— 

— 

36 (8.3) O 

33 (7.1) O 

MAT 

— 

39 (12.2) O 

29 (9.7) O 

— 

— 

30 (7.8) O 

27 (7.4) O 

SCI 

— 

50 (13.1) O 

39 (10.2) O 

— 

— 

40 (8.6) O 

36 (7.9) O 

Home Background Control Variables 








Students within Schools 










REA 

— 

— 

— 

15 (1.2) O 

15(1.2) O 

15 (1.2) O 

15(1.2) O 

Home Resources for Learning 

MAT 

— 

— 

— 

11 (1.0) 0 

11 (1.0) 0 

11 (1.0) 0 

11 (1.0) 0 


SCI 

— 

— 

— 

15 (1.1) O 

15 (1.1) O 

15 (1.1) O 

15(1.1) O 


REA 

— 

— 

— 

10 (0.8) O 

10 (0.8) O 

10 (0.8) O 

10 (0.8) O 

Early Literacy/Numeracy Tasks 

MAT 

— 

— 

— 

9 (0.8) O 

9 (0.8) O 

9 (0.8) O 

9 (0.8) O 


SCI 

— 

— 

— 

10 (1.1) 0 

10 (1.1) 0 

10 (1.1) 0 

10 (1.1) 0 

Between Schools 

School Average of 
Home Resources for Learning 

REA 

— 

— 

— 

67(3.3) O 

61 (3.3) O 

61 (4.0) O 

56 (4.1) O 

MAT 

— 

— 

— 

60 (3.2) O 

54(3.2) O 

54 (3.9) O 

50 (4.0) O 

SCI 

— 

— 

— 

64 (3.5) O 

57 (3.6) O 

57(4.2) O 

52 (4.3) O 

School Average of 

Early Literacy/Numeracy Tasks 

REA 

— 

— 

— 

32 (10.8) O 

33 (10.7) O 

22 (10.5) O 

24 (10.3) O 

MAT 

— 

— 

— 

23 (10.2) O 

24 (10.3) O 

14 (9.9) 

16 (10.0) 

SCI 

— 

— 

— 

35 (11.9) O 

37 (11.9) O 

25 (11.3) O 

27 (11.3) O 


() Standard errors appear in parentheses. REA -Reading © Coefficient significantlygreaterthan zero. 

MAT - Mathematics ® Coefficient significantly less than zero. 

SCI -Science 


Source of Variance 

| Percentage of Variance Explained 

ij School Explanatory Models |j 

Home 

| School Explanatory with Home Background Control Models 


School 

Environment 

School 

Instruction 

School Environment 
and Instruction 

Background 
Control Model 

School 

Environment 

School 

Instruction 

School Environment 
and Instruction 


Reading 


Between Schools (51%) 

25 

35 

49 

70 

74 

76 

79 

Within Schools (49%) 

— 

— 

— 

9 

9 

9 

9 

Total 

13 

18 

25 

40 

42 

43 

44 

Mathematics 

Between Schools (52%) 

24 

33 

46 

68 

72 

73 

76 

Within Schools (48%) 

— 

— 

— 

8 

8 

8 

8 

Total 

12 

17 

24 

39 

41 

41 

43 

Science 

Between Schools (49%) 

25 

36 

49 

65 

70 

73 

76 

Within Schools (51%) 

— 

— 

— 

9 

9 

9 

9 

Total 

12 

18 

24 

37 

39 

41 

42 


( ) Percentage of available variance shown in parentheses. 


TIMSS & PIRLS 

International Study Center 

Lynch School of Education, Boston College 


EFFECTIVE SCHOOLS IN READING, 
MATHEMATICS, AND SCIENCE AT THE FOURTH GRADE 

CHAPTER 3 177 


SOURCE: lEA's Trends in International Mathematics and Science Study and Progress in International Reading Literacy Study - TIMSS and PIRLS 201 1 



























References 

Creemers, Kyriakides, L., & 

Sammons, P. (2010). Methodological 
advances in educational effectiveness 
research. New York, NY: Routledge. 

Darling-Hammond, L. (2000). Teacher 
quality and student achievement: 

A review of state policy evidence. 
Education Policy Analysis Archives, 8(1). 
Retrieved from http://epaa.asu.edu/epaa/ 
vioni2/ 

Martin, M.O., Mullis, I.V.S., Foy, P, 

& Stanco, G. (2012). TIMSS 2011 
international results in science. Chestnut 
Hill, MA: TIMSS & PIRLS International 
Study Center, Boston College 

McGuigan, L. & Hoy, W.K. (2006). 

Principal leadership: Creating a culture 
of academic optimism to improve 
achievement for all students. Leadership 
and Policy in Schools, 5(3), 203-229. 

McLaughlin, M., McGrath, D.J., Burian- 
Fitzgerald, A., Lanahan, L., Scotchmer, 
M., Enyeart, C., & Salganik, L. (2005). 
Student content engagement as a 
construct for the measurement of effective 
classroom instruction and teacher 
knowledge. Retrieved from http:// 
www. air. org/files/ AER A20 05 Student, 
Content_Engagementn.pdf 

Mullis, I.V.S., Martin, M.O., Kennedy, A.M., 
Trong, K.L., & Sainsbury, M. (2009). 
PIRLS 2011 assessment framework. 
Chestnut Hill, MA: TIMSS & PIRLS 
International Study Center, Boston 
College 

Mullis, I.V.S., Martin, M.O., Foy, R, & 
Drucker, K.T. (2012). PIRLS 2011 
international results in reading. Chestnut 
Hill, MA: TIMSS & PIRLS International 
Study Center, Boston College 

Mullis, I.V.S., Martin, M O., Ruddock, G.J., 
O’Sullivan, C.Y., & Preuschoff, C. (2009). 
TIMSS 2011 assessment frameworks. 
Chestnut Hill, MA: TIMSS & PIRLS 
International Study Center, 

Boston College 


Mullis, I.V.S., Martin, M.O., Foy, P, & Arora, 
A. (2012). TIMSS 2011 international 
results in mathematics. Chestnut Hill, 

MA: TIMSS & PIRLS International Study 
Center, Boston College. 

Raudenbush, S.W. & Bryk, A.S. (2002). 
Hierarchical linear models (Second 
Edition). Thousand Oaks, CA: Sage 
Publications. 

Rumberger, R.W. & Palardy, G.J. 

(2004). Multilevel models for school 
effectiveness research. In D. Kaplan 
(Ed.), The sage handbook of quantitative 
methodology for the social sciences (pp. 
235-258). Thousand Oaks, CA: Sage 
Publications, Inc. 

Sammons, P. (2007). School effectiveness 
and equity. Making connection, a review 
of school effectiveness and improvement 
research and its implications for 
practitioners and policy makers. Reading, 
UK: CfBT Education trust. 

Scheerens, J. & Bosker, R.J. (1997). The 
foundations of educational effectiveness. 
Oxford, England: Pergamon. 

Teddlie, C. (2010). The legacy of the school 
effectiveness research tradition. In A. 
Hargreaves, E. Liberman, M. Fullan, & 

D. Hopkins (Eds.), Second international 
handbook of educational change (pp. 
523-554). New York, NY: Springer. 

Teddlie, C. & Reynolds, D. (2000). The 
international handbook of school 
effectiveness research. New York, NY: 
Falmer Press. 

Townsend, T. (Ed.). (2007). International 
handbook of school effectiveness 
and improvement. Dordrecht, The 
Netherlands: Springer. 


TIMSS AND PIRLS 201 1 RELATIONSHIPS REPORT 

178 CHAPTER 3 



TIMSS & PIRLS 

International Study Center 

Lynch School of Education, Boston College 




Chapter 4 


Effects of Home Background on Student 
Achievement in Reading, Mathematics, 
and Science at the Fourth Grade 


Jan-Eric Gustafsson, Kajsa Yang Hansen, and Monica Rosen 
University of Gothenburg, Sweden 


Introduction 

One of the most stable and consistently observed phenomena 
in the field of education is the impact of students’ home 
background on achievement. Students whose parents have a 
higher level of education, a more prestigious occupation, or 
greater income tend to have higher achievement than students 
whose parents have a lower standing on such socio-economic 
status (SES) indicators (e.g., Sirin, 2005). Many theories have 
been proposed to account for this phenomenon, but there is 
little consensus about which explanation is the most powerful. 
One reason is that, in spite of the stability of the phenomenon, 
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there is also considerable variation in strength of effects across educational 
systems and learning domains (Barone, 2006). So far there has been little 
research on this variation, and on the mechanisms which give rise to it. 

Gender is another student characteristic which tends to be related to 
achievement differences. However, here too considerable variation can be 
observed across learning domains, student age, and countries, and the nature 
and reasons for this variation is not well understood 

Taking advantage of the opportunities offered by the data collected in 
TIMSS and PIRLS 2011 at the fourth grade, the main aim of the research 
reported in this chapter is twofold: first, to describe the patterns of variation 
across countries and domains of learning (i.e., reading, mathematics, and 
science) in the relationship between student background characteristics and 
achievement; and second, to gain insight into some of the mechanisms which 
generate these relationships. A crucial design characteristic of the TIMSS 
and PIRLS 2011 fourth grade data used in this study is that the students were 
assessed in all three domains of learning — reading, mathematics, and science — 
which allowed simultaneous analysis of outcomes in domains where both 
literacy and numeracy skills are essential. A second important design feature 
of these data is that the students’ parents were asked to supply information in 
a Home Questionnaire about, among other things, different kinds of activities 
with the child, the child’s numeracy and literacy skills, and resources in the 
home. Given that home factors are likely to exert much of their influence before 
the start of formal schooling, the information in the Home Questionnaire is 
essential for understanding the mechanisms through which factors such as 
parental education and student gender influence school achievement. A third 
important design feature of the TIMSS and PIRLS 2011 data is the number 
of countries that participated. Altogether, 34 countries and 3 benchmarking 
entities took part in the study, enabling investigation of differences in the impact 
of home background factors on student achievement across a wide variety of 
school systems and cultures. 

In analyzing these data we have adopted a path modeling approach in 
order to investigate how the effect of parental education and gender on children’s 
achievement is mediated via the availability of home resources, early literacy 
and numeracy activities in the home, and literacy and numeracy skills when 
beginning school. Based on theoretical expectations and previous empirical 
results, we have constructed a model in which these hypothesized determinants 
have been included in chronological order. By estimating the strength of the 
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paths between these factors, and their direct and indirect effects on achievement 
in reading, mathematics and science at the fourth grade, this study seeks to 
gain a better understanding of the mechanisms through which educational 
inequalities are reproduced. 

This study has two main aims: to investigate to what extent parental 
education and gender influence fourth grade student achievement in 
reading, mathematics, and science in different countries; and to investigate 
the mechanisms through which parental education and gender influence 
achievement via books in the home, literacy and numeracy activities, and the 
child’s ability to carry out literacy and numeracy tasks when starting school. 

Relationships Between Student Background Factors 
and Achievement 

Typically, the correlation between SES and student achievement is about .30 
at the individual student level (Sirin, 2005; White, 1982). However, SES is a 
complex and multidimensional concept. The term cultural capital has been used 
to label the most important dimension of SES-influence on achievement. In 
most countries, parents’ formal educational level has been identified as the key 
component of cultural capital (Yang, 2003). 

One theoretical framework which is often used to explain the effect of 
parental education on achievement is Bourdieu’s Cultural Capital Theory 
(Bourdieu & Passeron, 1977). This theory basically argues that social classes 
preserve a strong cultural identity, and that social origins have a strong influence 
on students’ cultural resources. Skills, attitudes, and use of language, to take a 
few examples, thus are differentiated according to class origins. Furthermore, 
pedagogical practices and assessment procedures are to a large extent related to 
the culture of the upper class, which contributes to making cultural capital the 
main determinant of school and occupational success. 

Barone (2006) used PISA 2000 data from 25 countries to test the Cultural 
Capital Theory, using indices of cultural capital from the PISA questionnaire 
and SES and parental education as indicators of social class. Barone concluded 
that the indicators of family cultural capital had only modest explanatory power, 
and observed that the effects of these variables may be better interpreted as 
indirect signs of the importance of cognitive resources. He also suggested 
that the limited explanatory power of Cultural Capital Theory may be due to 
the existence of other causal mechanisms that mediate the influence of social 
origins, such as occupational ambitions. 
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Influences of Home Environment on Child Development 

Much research has focused on what is important for developing children’s 
language and cognitive skills that can ultimately lead to educational success 
(Park, 2008). One such factor is parental reading habits, which can create a 
favorable reading climate (De Graaf, De Graaf, & Kraaykamp, 2000). A great 
deal of research on child development, especially in the United States, also 
has highlighted the importance of home literacy environments that stimulate 
the development of the child’s cognitive and language skills (e.g., Farkas & 
Beron, 2004). Researchers have found substantial differences in home literacy 
environments of children from high and low SES families, which in turn explain 
educational differences between the two groups of children (Brooks-Gunn, 
Klebanov, & Duncan, 1996; Duncan, Brooks-Gunn, & Klebanov, 1994). In the 
following section we review some of the main lines of research on development 
of literacy and numeracy skills. 

HOME ENVIRONMENT AND THE ACQUISITION OF LITERACY SKILLS In order to understand 
how parental education and gender influence the development of early literacy 
and numeracy skills, it is useful to take as a starting point what is known about 
the general mechanisms and factors which are important for the acquisition 
of these skills. Much more research has been conducted on literacy than on 
numeracy skills, so we begin with the literacy research. 

The U.S. National Early Literacy Panel (2008) has conducted a research 
synthesis in the form of multiple meta-analyses of approximately 500 empirical 
early literacy studies. The synthesis, titled Developing Early Literacy, identified 
six variables as being important precursors and predictors of reading skills, 
including the following: alphabet knowledge; phonological awareness (the 
ability to detect, manipulate, or analyze the auditory aspects of spoken language 
independent of meaning); ability to write letters in isolation or write one’s name; 
phonological memory (the ability to remember spoken information for a short 
period of time); and rapid automatized naming of letters/digits and of objects/ 
colors. 

Additional meta-analyses included in Developing Early Literacy focused on 
the effects of different types of interventions in determining the effectiveness 
of instructional strategies, programs, or practices in teaching literacy skills or 
the precursor skills. For example, code-focused interventions are designed 
to teach skills related to cracking the alphabetic code, and typically included 
phonological awareness instruction. This type of intervention yielded moderate 
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to large effects on the predictors of literacy and on conventional measures 
of literacy. Shared reading interventions basically involved reading books 
to children. Book-sharing interventions produced moderate-size effects on 
children’s oral language skills and print knowledge. There were no differences 
in the effects of shared reading based on whether parents or teachers did the 
reading. 

Parent and home programs interventions use parents as agents of 
intervention and include interventions that teach parents instructional 
techniques for use with their children at home. These interventions yielded a 
moderate to large effect on oral language outcomes and on general cognitive 
abilities. However, the design of the programs varied greatly, with some having 
general goals of improving children’s health, behavior, or cognitive functioning, 
and others more specific goals such as improving children’s oral language 
skills. Language enhancement interventions examined the effectiveness of 
instructional efforts aimed at improving young children’s language development. 
These interventions succeeded in increasing children’s oral language skills to a 
large degree. 

There was little evidence that literacy interventions were differentially 
effective in terms of gender or SES. However, this may be because few studies 
reported the results of such interactions. 

The findings from these meta-analyses of interventions suggest that 
parents and preschools can influence the literacy development of young 
children. These studies show that learning resulted from teaching children 
phonological awareness, reading to the child, involving parents in their 
children’s learning, and teaching oral language skills. The fact that these effects 
have been demonstrated with experimental designs and systematic syntheses 
of findings is important, because this makes inferences about causality credible. 
The problem of explaining why parental education and gender is associated 
with educational achievement cannot easily be approached with experimental 
designs, but in cross-sectional surveys we can take advantage of results based on 
experimental studies. Thus, if it can be demonstrated that parents with a higher 
level of education to a larger extent are involved in activities and practices that 
have been shown through experimental work to have positive effects on literacy 
development than are parents with a lower level of education, this provides 
support for an explanation of the effects of parental education as being mediated 
by these activities and practices. 
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As has already been pointed out, a large body of literature has demonstrated 
strong effects of SES, in particular parental education, on the reading skills and 
academic achievements of the child (e. g. Davis-Kean, 2005; Hecht, Burgess, 
Torgesen, Wagner, & Rashotte, 2000; Lyster, 2002; Myrberg & Rosen, 2009; Raz 
& Bryant 1990; Senechal & LeFevre, 2002). In general, this relationship has been 
attributed to parents’ beliefs, values, expectations, attitudes and behaviors: well 
educated parents appear to have high expectations of their children, while at 
the same time adapting their expectations to the performance of their children. 
In contrast, parents with little education tend to have lower, or sometimes 
unrealistically high, expectations of their children. Also, high parental education 
is related to a warm, social climate in the home (Duncan, Brooks-Gunn & 
Klebanov, 1994). 

Along similar lines, parents with higher education tend to interact more 
verbally with their child; they use more abstract words, more complex syntax, 
and invite their child more often into decontextualized discourse (Bernstein, 
1971), book-sharing, and dialogical reading (Jordan, Snow, & Porsche, 2000). 
These language practices mirror the language of books and school and foster 
good literacy skills (Tabors, Snow, & Dickinson, 2001). 

Senechal and LeFevre (2002) found that informal shared reading of 
storybooks during preschool years seemed unrelated to parents’ teaching of 
reading. The authors demonstrated in a longitudinal study that different types 
of activities also are associated with different outcomes. The link between 
parents’ reports of teaching reading and reading storybooks with children was 
indirect and mediated through children’s emergent literacy skills. The variables 
that were directly related to reading skills at the end of the first grade were 
those most closely tied to the mechanics of reading. However, the pathways 
for reading achievement in the third grade were different. Reading storybooks 
at home predicted children’s receptive language skills both concurrently 
and longitudinally. Senechal and LeFevre found that children’s exposure to 
storybooks at home began to show a strong link to reading performance once 
the mechanics of reading were under control and children were reading fluently. 
Their results thus indicate that children must acquire sufficiently fluent decoding 
skills before receptive language skills can exert their full influence. 

As noted by Snow, Burns, and Griffin (1998), vocabulary knowledge has 
been shown in several studies to be a major correlate of reading comprehension, 
and comprehension is diminished by lack of relevant word knowledge. Hart 
and Risley (1995) studied vocabulary development of one- to three-year-olds 
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as related to parental communication patterns. Parents with an academic 
background made use, on average, of three times as many words per hour as 
parents on welfare, and their children’s vocabulary development appeared to 
mirror this difference: by age three the children in “the academic group” had 
a vocabulary of 1500 versus 500 in the “welfare group.” The authors argued 
that differences in parental language pattern contributed to a “language gap” 
between children from high and low social classes of many thousands of words 
at later ages. 

The meta-analyses included in Developing Early Literacy also suggest that 
phonological awareness is causally related to early reading acquisition. Raz and 
Bryant (1990) concluded on the basis of a longitudinal study that SES differences 
in decoding skills can be entirely explained by the influences of SES factors 
on phonological awareness. Hecht, Burgess, Torgesen, Wagner, and Rashotte 
(2000) also found that social class differences in early reading acquisition could 
partly be accounted for by differences in phonological abilities and that levels 
of print knowledge (i.e., knowledge about books and reading) to a large extent 
accounted for SES differences. 

Based on the literature reviewed above, it is reasonable to hypothesize 
that SES effects on reading acquisition are mediated via both phonological and 
vocabulary skills, and that the SES effects are largely caused by variations in 
experiences of language and text (Noble, McCandless, & Farah, 2007). Taken 
together, these studies emphasize the importance of both the volume and quality 
of verbal activities and interactions in the home. 

HOME ENVIRONMENT AND THE ACQUISITION OF NUMERACY SKILLS Compared to the 
number of research studies on early literacy, very few have been conducted on 
early numeracy, and even fewer have simultaneously investigated early literacy 
and early numeracy. One reason for this is that early numeracy (also referred to 
as quantitative literacy, or mathematical literacy) is more difficult to define than 
reading literacy. While there is consensus that number skills form an important 
aspect of numeracy, many researchers offer a broader view of the nature of 
numeracy. Thus, Diezman and Yelland (2000) argue that the foundational 
processes of numeracy are representation, manipulation, reasoning, and 
problem solving. Classification of objects and shapes, estimating, measuring, 
and reproducing number patterns are other examples of skills associated with 
numeracy (Ewers-Rogers & Cowan, 1996). Also, literacy and numeracy often 
are intertwined (Aiken, 1972). This maybe a reason why there are few programs 
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that are intended to support parental promoting of numeracy development for 
their children. Furthermore, it has been argued that any program developing 
language and problem-solving skills at young age will have consequential 
numeracy effects (Doig, McCrae, & Rowe, 2003). 

Anders et al. (2012) report a study which investigated the domain 
specificity of numeracy and literacy stimulation in home and preschool settings 
in order to disentangle the effects of the two domains. They argued that it is 
reasonable to assume that numeracy-related activities and stimulation are 
especially beneficial for the development of numeracy skills. However, they also 
recognized that verbal and pre-reading related activities and stimulation may 
foster the development of numeracy skills. In a longitudinal study, they followed 
a sample of 532 children attending 97 preschools from ages 3 to 5. There were 
three waves of measurement at which information about the children’s verbal 
and numeracy skills were collected, along with detailed information about, 
among other things, literacy- and numeracy-related activities in the home, and 
measures of preschool structural and process quality. 

The study combined interviews and questionnaires with observations 
in the family setting. Using information from these sources, a literacy scale 
containing the following ten items was constructed: toys for free expression, 
number of children’s books, books in the household, stimulation to learn the 
alphabet, stimulation to learn to read, questions in interaction, amount of free 
discussion, interactions regarding letters, phonological cues, and frequency of 
shared book reading. A numeracy scale consisting of the following ten items 
also was constructed: toys to teach colors and shapes, toys to teach numbers, 
stimulation to learn shapes, stimulation to learn colors, stimulation to learn 
spatial relationships, stimulation to learn digits, stimulation to learn counting, 
interaction regarding digits, interaction regarding shape and space, and 
interaction regarding comparing and classifying. The correlation between the 
two scales was r = 0.62, indicating a moderate degree of relationship. 

The data were used to investigate several research questions, but 
this chapter focuses on results pertaining to effects of the home learning 
environment on numeracy development. Growth curve modeling was used 
as the main analytic method. First age, and a set of background variables, 
were included in the model, and then the literacy and numeracy indicators 
were included separately as additional predictors. The quality of the home 
learning environment explained substantial variance in numeracy at the first 
assessment, but there was no significant effect of home learning environment 
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on development after the first assessment. The results also showed that the 
quality of the home environment in terms of promoting literacy skills was more 
strongly correlated with initial numeracy skills than was the quality of the home 
environment in terms of promoting numeracy skills. 

In addition, the results showed that the influence of maternal educational 
level and SES decreased when home learning environment was included in 
the model, suggesting that part but not all of the relationship between family 
background and numeracy is explained by the quality of the home learning 
environment. This effect was more pronounced for literacy environment than 
for numeracy environment. 

The study thus showed that the effect on numeracy skills was stronger for 
quality of literacy stimulation than it was for quality of numeracy stimulation. 
This was contrary to expectations, and Anders et al. (2012) observed that one 
reason for this may be that the assessment used to measure numeracy skills 
required not only numeracy but also language skills. They also argued that 
adequate language skills are a prerequisite for the acquisition of mathematical 
knowledge; thus, the quality of the home learning environment with respect 
to verbal literacy at this early age may have more impact than its quality to 
promote numeracy. Another possible interpretation was that the literacy scale 
captured more general beneficial characteristics of home learning environment 
(e.g., routines) than the numeracy scale. The relative rarity of numeracy-related 
resources and parental activities also was noted as a possible contributory factor. 

GENDER DIFFERENCES The pattern of gender differences in achievement in 
mathematics and science varies as a function of the age of the students. In 
analyses of the TIMSS 1995 data, Mullis, Martin, Fierros, Goldberg, and Stemler 
(2000) found few differences in average mathematics achievement at the fourth 
and eighth grades, but substantial differences at the twelfth grade. A similar 
pattern of results was found for science, although gender differences already 
were present in many countries by the fourth grade. 

Other studies also have demonstrated that a male performance advantage 
in mathematics and science achievement emerges only after elementary school 
and that it grows larger with increasing age (see Spelke, 2005, for a review). 
Furthermore, meta-analyses have revealed that most gender differences in 
cognitive abilities underpinning achievement in these areas are small (Hyde, 
2005). In a review of the literature, Spelke (2005) concluded that male and 
female infants do not differ in the cognitive abilities that form the foundations 
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of mathematical and scientific thinking, and that male and female children 
master the concepts and operations of elementary mathematics in the same 
way at the same time. 

Baker and Jones (1993) proposed a gender stratification hypothesis 
to account for observed gender differences in mathematics and science 
achievement in the higher grades. The gender stratification hypothesis holds 
that, in patriarchal cultures, the achievement of male students is linked to 
their future opportunities. Female students see mathematics and science as 
less important for their future and are socialized into this mode of thinking 
by teachers, parents, and friends. Thus, according to the gender stratification 
hypothesis, opportunity structures shape socialization processes that shape 
performance. Furthermore, the hypothesis proposes that where there is more 
societal stratification based on gender, females will perform less well on 
mathematics and science achievement tests than will males. 

Else-Quest, Hyde, and Finn (2010) describe various psychological theories 
that identify socialization processes accounting for the effects of gender 
stratification. One of these is the expectancy- value theoretical model proposed 
by Eccles (1994) to explain the gender gap in mathematics achievement, 
attitudes, and the underrepresentation of women in fields such as science and 
engineering. According to this model, people need to value a task to undertake 
it, and they need to have some expectation of success. Perceptions of the 
task’s value are influenced by, among other things, the culture and cultural 
stereotypes related to gender and by the person’s short-term and long-term 
goals. Expectations of success are influenced by self-concept, which in turn are 
influenced by parents’ and teachers’ attitudes and expectations, which often are 
gender stereotyped. 

Bandura’s (1986) cognitive social learning theory also identifies social 
processes that contribute to the development of gender-typed behavior. 
According to this theory, role models, socializing agents, and perceptions of 
gender- appropriate behavior influence an individual’s actions and choices. Like 
the expectancy-value theory, this theory emphasizes the role of self-efficacy 
in gender-typed behaviors. The theory proposes that girls are attentive to the 
behaviors that women in their culture engage in, and thus feel efficacious in 
and model those behaviors. In its emphasis on observational learning and the 
internalization of cultural norms, the cognitive social learning theory provides 
an individual-level explanation of why girls act in ways that reproduce societal- 
level gender stratification. 
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Given that the students participating in the TIMSS and PIRLS 2011 fourth 
grade assessment were still quite young (around 10 years old), we do not expect 
any large gender differences to be seen in mathematics and science achievement. 
However, gender differences in reading achievement at this level have 
consistently been found in international assessments (see, e.g., Mullis, Martin, 
Gonzalez, & Kennedy, 2003; Mullis, Martin, & Kennedy, 2007; Mullis, Martin, 
Foy, & Drucker, 2012). Research also consistently identifies gender differences 
in attitudes to reading and in reading motivation. Ming Chui and McBride- 
Chang (2006) analyzed gender differences in reading comprehension in 43 
countries participating in PISA with samples of 15-year-olds and concluded that 
girls outperformed boys in each and every country. However, even though the 
size of the gender difference varied across countries, it proved difficult to find 
variables that mediated the gender difference. Reading enjoyment did mediate 
the difference to some extent, but this variable could be seen as another outcome 
variable rather than as an explanatory variable. 

It is reasonable to expect that gender differences in reading achievement 
are partly due to differential opportunities for boys and girls to acquire early 
literacy skills in the home and preschool. Thus, if it can be demonstrated that 
girls are more involved in activities and practices shown to have positive effects 
on literacy development than boys, this can explain some of the observed gender 
differences in reading achievement. 

Results from Previous PIRLS Path Analyses 

Given that a Home Questionnaire has been available since the first PIRLS 
assessment in 2001, there have previously been opportunities for analyzing 
determinants of reading literacy with path modeling techniques. Park (2008) 
used data from PIRLS 2001 to compare the ways in which home literacy 
environment influence reading achievement at the fourth grade in 25 countries. 
Three measures were used as indicators of home literacy environment: Early 
Home Literacy Activities Index, which is an average of six items; Number of 
Books at Home; and Parents Attitudes Toward Reading, which is an index based 
on four items. Ordinary least squares regression models were developed for 
each country separately, in which the effects on reading achievement of these 
three home literacy variables were estimated. A second series of OLS-models 
investigated the extent to which the home literacy variables mediated the effect 
of parental education by comparing the gap in reading score between students 
from high and low parental education groups in models with and without the 
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three variables. Park reported small mediating effects of early home literacy 
activities, as it did not reduce the difference between students from the parental 
education groups by more than 10 percent in any country. The reduction 
remained modest (20-30%) for most countries also when all three home literacy 
environment variables were included in the model, even though the reduction 
exceeded 50 percent for some countries. Separate analyses indicated that Early 
Home Literacy Activities and Parental Attitudes toward Reading had smaller 
effects than did Number of Books at Home in 20 of 25 countries. One reason 
for this maybe that the activities and attitudes indices were more influenced by 
errors of measurement than was the Books variable. 

Myrberg and Rosen (2009) used data from the Swedish participation in 
PIRLS 2001 to estimate the effect of parents’ education on children’s reading 
achievement, and to estimate the indirect effects of different mediating factors. 
Effects of parental education were hypothesised to be mediated through the 
number of books in the home, via early reading activities with the children 
during the preschool years and via the children’s early reading abilities. 

The study made use of structural equation modelling with latent 
variables. In the first step, the measurement model was created, in which the 
latent variables were defined in terms of their relation to observed variables. 
The measurement model included four latent variables: Parental Education, 
with mother’s education and father’s education as indicators; Books at Home, 
measured by number of books in the home and by number of children’s books 
in the home; Early Reading Activities, measured by two items from the Home 
Questionnaire (read with child, and tell stories to child); and Early Reading 
Abilities at School Start, measured by three items (recognize letters, read words 
and read sentences). 

Based on the measurement model, the path model specified how the latent 
variables were expected to influence each other and the reading achievement 
outcome variable. The latent variables were ordered chronologically and logically 
as follows: Parental Education preceded Books at Home, which preceded the 
Early Reading Activities with the preschool child, which preceded Early Reading 
Abilities at School Start (the child’s emergent literacy at the beginning of first 
grade), which preceded the PIRLS reading achievement score. 

While the direct effect of Parental Education on reading achievement was 
modest (.17), the total effect was substantial (.34). This estimate agreed with 
what has been found in previous research (White, 1982; Yang, 2003). The total 
indirect effect, which is the difference between the total effect and direct effect, 
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thus accounted for about 50 percent of the total effect. The strongest indirect 
effects went via Books at Home, of which the most important was directly from 
Books at Home to achievement. Two minor indirect effects were mediated 
through Early Reading Activities, one directly to reading achievement and 
one via Early Reading Abilities. Finally, there was an indirect effect of Parental 
Education via Early Reading Abilities. The model thus explains a part of the 
effect of parents’ education on achievement in terms of books at home and use 
of those books for literacy purposes. 

Method 

The model applied in this study is an extension of the Myrberg and Rosen 
(2009) model. Besides parental education, the present study includes gender as 
an independent variable. Also, the TIMSS and PIRLS 2011 Home Questionnaire 
inquired about numeracy and literacy activities in addition to numeracy and 
literacy skills when beginning primary school; therefore, in this extended model, 
the numeracy-literacy distinction is central. Furthermore, the extended model 
includes three outcome achievement variables: reading, mathematics, and 
science. 

The Modeling Approach 

This study aims to investigate the effects of the two independent variables — 
parental education and gender — on the three dependent variables — reading, 
mathematics, and science achievement — allowing for the possibility that 
there are both direct relationships between the independent variables and 
the dependent variables, and relationships involving other variables, which 
simultaneously behave as independent and dependent variables. 

We may, for example, hypothesize that one reason why we observe a 
relationship between parental education and reading achievement is that 
the frequency of literacy activities (LitAct) is higher in homes with more 
highly educated parents than in homes where the parents have a lower level 
of education. Another way to express this is to say that parental education 
influences LitAct, which in turn influences reading achievement. In Exhibit 
4.1, two simple models are shown: Model A, and Model B. In Model A, there is 
a direct relationship between Parental Education and Reading. The regression 
coefficient (bl) expresses the “direct” effect of Parental Education on Reading 
achievement. In this model, the direct effect also is the “total” effect because the 
regression expresses the maximum linear relationship between the independent 
and the dependent variables. 
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Exhibit 4.1 : Two Path Models for Relationships between Parental Education and 

Reading Achievement 

Model A 


Parental 

education 


bl 


Reading 


Model B 



In Model B there is a path between Parental Education and LitAct, with 
regression coefficient b2, and also a path between LitAct and Reading (b3). 
These two relationships constitute an indirect relationship between Parental 
Education and Reading and the product of b2 and b3 represents the strength of 
this relationship. In Model B there also is a direct relationship between Parental 
Education and Reading achievement (bl '). The coefficient bl ' is not the same 
as coefficient bl in Model A because bl = bl '+ b2b3. This means that the total 
effect of Parental Education on Reading (i.e., bl) can be decomposed into one 
direct effect (bl ') and one indirect effect (b2b3). If b2 and b3 both are positive 
(which of course is not necessarily the case), bl ' will be smaller than bl. In 
substantive terms, Model B partially explains the relationship between Parental 
Education and Reading achievement in terms of a mediating mechanism, 
through which parents with higher levels of education involve their children in 
literacy activities to a larger extent than parents with lower levels of education, 
and these literacy activities in turn have a positive effect on reading achievement. 

The mediating effect may account for only a part of the total effect, in 
which case further mediating variables and mechanisms might be sought for. 
It may also be that the indirect effect is as large as the total effect, so that there 
is no direct effect. This is referred to as “complete mediation.” 

This simple example describes the general principles for distinguishing 
between total, direct, and indirect effects, which we apply in this study in 
analyzing the effects of parental education and gender on fourth grade student 
achievement. 
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The extension of the Myrberg and Rosen (2009) model described above 
has guided the construction of the models that we have tested against the data 
in this study. The translation of a conceptual model to a path model that can be 
estimated and tested empirically involves several steps. The first step is to specify 
the variables to be included in the model and the second step is to propose a 
hypothesized path model. In the third step, the model is estimated from data 
and the goodness-of-fit of the model is evaluated. The fourth and final step is 
to compute the total and indirect effects, and to interpret these. Each of these 
steps are described in the following section. 

Developing the Measurement Model 

The Home Questionnaire inquires about both numeracy and literacy activities 
in the home, and about the child’s abilities in performing numeracy and literacy 
tasks. The starting point for the selection of items to be included in the analysis 
was the items in the four TIMSS and PIRLS 2011 background scales: Early 
Literacy Activities Before Beginning Primary School (9 items), Early Numeracy 
Activities Before Beginning Primary School (6 items), Could Do Early Literacy 
Tasks When Began Primary School (5 items), and Could Do Early Numeracy 
Tasks When Began Primary School (6 items). The items in these scales are 
presented in Exhibit 4.2. 

Exhibit 4.2: Items in the Scales Measuring Literacy and Numeracy Activities, and 

Literacy and Numeracy Skills at Start of School 

Items in the Early Litreracy Activities Before Beginning Primary School Scale 

Before your child began primary/elementary school, how often did you or someone else in your 
home do the following activities with him or her? 

Often Sometimes Never or 


almost never 

1 1 i 

1) Read books Q 

3) Sing songs O — O O 

4) Play with alphabet toys (e.g., blocks with letters of the alphabet) Q <- ■» Q Q 

5) Talk about things you had done Q Q Q 

6) Talk about what you had read O O O 

7) Play word games Q O O 

8) Write letters or words Q Q Q 

9) Read aloud signs and labels Q Q Q 


EFFECTS OF HOME BACKGROUND ON STUDENT ACHIEVEMENT IN 
TIMSS & PIRLS READING, MATHEMATICS, AND SCIENCE AT THE FOURTH GRADE 

International Study Center 

Lynch School of Education, Boston College v H A r I tK 4 


195 



Exhibit 4.2: Items in the Scales Measuring Literacy and Numeracy Activities, and 

Literacy and Numeracy Skills at Start of School (Continued) 

Items in the Early Numeracy Activities Before Beginning Primary School Scale 

Before your child began primary/elementary school, how often did you or someone else in your 
home do the following activities with him or her? 

Often Sometimes Never or 

almost never 

▼ v i 

1) Say counting rhymes or sing counting songs Q Q Q 

2) Play with number toys (e.g., blocks with numbers) Q Q Q 

3) Count different things O O O 

4) Play games involving shapes (e.g., shape sorting toys, puzzles) Q Q O 

5) Play with building blocks or construction toys Q Q Q 

6) Play board games or card games Q Q Q 


Items in the Could Do Early Literacy Tasks When Began Primary School Scale 


How well could your child do the following when he/she began primary/elementary school? 


1 ) 

2 ) 

3) 

4) 

5) 


Recognize most of the letters of the alphabet 

Read some words 

Read sentences 

Write letters of the alphabet 

Write some words 


Very Moderately Not very Not 

well well well at all 


1 


I 






o 

o 

o 

o 

o 


o 

o 

o 

o 

o 

o 

o 

o 

o 

o 

O' 

o 

o 

O' 

o 


Items in the Could Do Early Numeracy Tasks When Began Primary School Scale 


Could your child do the following when he/she began primary/elementary school? 

Up to 100 Up to 20 Up to 10 Not at all 
or higher 

1) Count by himself/herself Q Q O O 

More than 3-4 shapes 1-2 shapes None 
4 shapes 

2) Recognize different shapes (e.g., square, triangle, 

circle) O' O O- 

All 10 5-9 1-4 


-o 

None 


3) Recognize the written numbers from 1-10 

4) Write the numbers from 1-10 


numbers numbers numbers 

-o -o -o -o 

-o- o o -o 

Yes No 


5) Do simple addition — 

6) Do simple subtraction - 


o 

o 


■o 

■o 
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These scales have been carefully constructed and their psychometric 
properties are well documented (Martin, Mullis, Foy, & Arora, 2012). However, 
the relatively limited number of items in the scales causes their reliabilities, 
which vary between .66 and .90, to be somewhat too low to be used as manifest 
variables in a path model. One solution to this problem could have been to 
define error-free latent variables with the items in the scales as indicators. 
However, this would have required 26 variables for this part of the model alone, 
which would have caused the complete model to be unwieldy and tedious to 
estimate. Instead, a compromise solution was adopted, where “testlets” were 
created by dividing the items in each scale into two random halves and using 
these as indicators of latent variables. 

A major advantage of using latent variables when investigating chains of 
relationships among different determinants is that the relationships are not affected 
by errors of measurement in the observed variables (see, e.g., Brown, 2006). 
However, latent variable models often are afflicted by other problems. A common 
problem encountered in application of latent variable models is multicollinearity, 
which occurs when two or more independent variables are highly correlated. In 
this situation, there is too little unique information available for each independent 
variable, making it impossible to achieve stable and interpretable estimates of the 
influence of these variables on the dependent variable. 

The fact that variables are sometimes difficult to separate from one another 
is, however, above all a conceptual problem. A common reason for overlap 
between observed variables is that, to a large extent, they measure the same 
underlying variable. For example, when parents estimate how often literacy and 
numeracy activities have taken place in the home, it may be that their responses 
reflect a general level of educationally-oriented activities with the child, rather 
than specifically whether the activities were of literacy or numeracy kinds. If 
that is the case, the literacy and numeracy scales would be highly correlated and 
multicollinearity problems would occur if both were used as independent variables. 

At the same time, because we may expect differences between families with 
respect to the general level of activities, it is also reasonable to expect that the 
balance of numeracy and literacy activities varies between families, such that in 
some families there is more of literacy activities than numeracy activities while 
in other families there is more of numeracy activities than literacy activities. 
There also may be differences between countries in these respects. A model with 
two correlated latent variables representing the amount of numeracy and literacy 
activity, respectively, allows us to determine the impact that the two types of 
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activity have on educational achievement. These impacts are determined in such 
a way that the effect of literacy activity is determined with the level of numeracy 
kept constant, and vice versa. However, with this approach to measurement it is 
not possible to see the effect of differences in level of activity on achievement, 
because these differences only affect the correlation between the two variables. If 
this correlation is high, the analysis will be affected by multicollinearity and we 
still will not be able to determine any effects of general level of activity, because 
general level of activity is not represented by any variable. 

While the traditional approach to measurement would suggest construction 
of two separate, but correlated, measures of literacy and numeracy activities, 
other approaches also are possible. In many fields of research, there is a need to 
identify both broader, more general aspects of phenomena, and more narrow 
or specific aspects (Gustafsson & Aberg-Bengtsson, 2010). Some examples of 
such fields are research on cognitive abilities, educational achievement, and 
personality, where it is easy to identify variables which have a broad scope of 
reference and variables which have a narrow scope of reference (Gustafsson, 
2002 ). 

Recently, special techniques have been developed for modeling data with 
latent variables of different degrees of generality. These modeling approaches are 
referred to as “bi-factor models” (e.g., Reise, 2012) or as “nested-factor models” 
(Gustafsson & Balke, 1993). With this approach, a general latent variable is 
typically identified for a domain of observations, along with narrow latent 
variables which account for observed differences on subsets of variables. 

Such a bi-factor modeling approach is suitable in this case because we are 
interested in determining the effects both of the general level of activities in the 
home, and of the balance between numeracy and literacy activities. A latent 
variable model has therefore been constructed in such a manner that there is one 
general activity variable (Activity), which is taken to be positively related to the 
four manifest testlet variables, and there is one latent variable which represents 
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a contrast between numeracy activities on the one hand and literacy activities 
on the other hand. This latent variable (NumLitAct) has fixed relationships of 
positive unity to the two testlets representing literacy activity (LITACTl and 
LITACT2) and fixed relations of negative unity to the two testlets representing 
numeracy activity (NUMACTl and NUMACT2). The NumTitAct variable 
thus represents the degree of balance between the two types of activity, with 
positive values indicating more literacy than numeracy activity and negative 
values indicating more numeracy than literacy activity 

A similar line of reasoning can be applied to the parents’ reports of how well 
the child could do various numeracy and literacy tasks before beginning primary 
school. Here, too, there is reason to expect a high level of correlation between the 
measures from the two domains, suggesting that a more appropriate approach 
would be to define one latent variable representing ability to do both kinds of 
tasks (Ability), and a second latent variable representing ability to do literacy 
tasks better than numeracy tasks (NumLitAb). These two latent variables were 
constructed in the same manner as the two activity latent variables. Thus, the 
NumLitAb variable was specified to have fixed relations of unity to the two testlets 
representing literacy skills (TITAB1 and TITAB2) and fixed relations of negative 
unity to the two testlets representing numeracy skills (NUMAB1 and NUMAB2). 
The Ability variable was specified to be related to all these four testlets. 

The measurement model included one additional latent variable 
representing literacy resources in the home. The Books latent variable had two 
indicators: the number of books in the home (NBOOK), and the number of 
childrens books in the home (NCBOOK), as reported by the parents. 

The model also included two independent variables, parental education 
and gender. Parental Education was defined as the highest level of education 
of either parent, and Gender was represented by a binary variable (boy = 0 and 
girl - 1). 
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The Hypothesized Path Model 

Exhibit 4.3 presents a schematic and highly simplified version of the 
hypothesized path model. 


Exhibit 4.3: A Schematic Description of the Hypothesized Path Model 



While Exhibit 4.3 presents the two independent variables (Parental 
Education and Gender) and all of the latent variables in the model, as well as the 
three dependent variables (achievement in mathematics, science, and reading). 
For clarity, the observed variables that serve as indicators of the latent variables 
are not shown. According to this model, the latent variable Books influences 
Activity and NumLitAct, and these in turn influence Ability and NumLitAb, 
respectively. The latter two latent variables are assumed to influence the three 
achievement variables. This model thus formulates the hypothesis that Parental 
Education and Gender influence the extent to which Books are available in 
the home, and that these in turn influence both the general level of Activity of 
educational tasks in the home and the balance between numeracy and literacy 
activities, which in turn influence the child's Abilities at the start of primary 
school. 

Compared to the model that was actually estimated, the model shown in 
Exhibit 4.3 presents only a small subset of the relationships among variables. 
The estimated model was a “saturated” model in which each variable in the 
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path model was related to every other variable to the right of it. Thus, each 
and every variable was predicted by Parental Education and Gender, Books 
predicted all the latent variables, and so on. Many of these direct effects were 
found to be non-significant, but no attempts were made to prune away non- 
significant relationships from the models. 

In addition to estimation of the direct effect of one variable on another, 
total and indirect effects were computed. The total effect of an independent 
variable on a dependent variable is the sum of the direct effect and of all 
indirect effects. A specific indirect effect is a function of the product of the 
path coefficients encountered along a particular route from the independent 
variable to the dependent variable. The total indirect effect is the sum of all 
possible specific indirect effects. 

Estimation 

In the first step of estimation, a single model based on the combined data from 
all 37 participants was fitted. This Common model was estimated using the two- 
level modeling technique available in the Mplus program (Muthen & Muthen, 
1998-2012), with country as the between -level, and students within country as 
the within-level. For the between-level, a saturated model was fitted which freely 
estimated the covariances among the country means. For the within-level, the 
saturated path model was fitted. This model thus was fitted to the pooled- within 
matrix for all the participating countries, which is not influenced by any mean 
differences across countries. 

In the next step of analysis, a separate model was fitted for each country. 
These models were estimated with the Mplus 7.11 program (Muthen & Muthen, 
1998-2012), using the MFR estimator. This estimator takes non-normality 
of the distributions of the observed variables into account, and corrects for 
the underestimation of standard errors that is caused by deviations from the 
assumption of multivariate normality that the maximum likelihood estimator is 
based upon. The so called “Complex option” in Mplus also was used, with school 
as the cluster variable, to correct for underestimated standard errors due to the 
cluster sampling techniques employed in drawing the samples in each country. 
In the analysis, individual student case weights (HOUWGT) were used. 

The analyses took into account all five plausible values (PVs) available for 
each of the three achievement measures by relying on the Mplus Imputation 
facility, which computes one analysis for each PV and then combines these into 
a single parameter estimate and a single estimate of the standard error. However, 
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this procedure was not available for the estimation of total and indirect effects; 
therefore, in order to obtain estimates based on all five PVs a special program 
written using the Model Constraints facility available in Mplus. All of the 
parameter estimates and standard errors presented in this chapter thus are based 
on five plausible values. 

The fit of the model to the data was evaluated with a set of tests and indices 
computed by the Mplus program. One basic source of information about the 
degree of fit of a model to data is the chi-square goodness-of-fit test, which for 
a well-fitting model should be non-significant. Mplus computes the test once 
for each plausible value, and reports the mean and standard deviation of the 
five results. A difficulty with the chi-square test statistic is that it increases as a 
function of sample size; therefore, given the large number of observations in our 
data, the test is practically always significant, indicating that the model should 
be rejected as not fitting the data. However, this is because the large number of 
observations provides statistical power to detect even trivial deviations between 
the model and data. Thus, for these analyses there was a need for indices of fit 
that provide information about the degree of deviation between the model and 
data. 

One such measure is the Root Mean Square Error of Approximation 
(RMSEA), which indicates the degree of deviation between model and data, 
taking into account both model complexity and sample size. RMSEA should be 
as low as possible, preferably lower than .05 (or .07-.08). Another useful measure 
is the Comparative Fit Index (CFI), which should be higher than .95 and as close 
to unity as possible. The Standardized Root Mean Square Residual (SRMR) 
measures the amount of deviation between the elements of the observed 
covariance matrix and the model-implied matrix, and according to the rules of 
thumb this measure should be lower than .08. 

Descriptive Statistics 

Exhibit 4.4 presents the means and standard deviations of the independent and 
mediating variables used in the analyses. The variables have been coded in such 
a way that higher values imply a higher level on the dimension measured, except 
for the dummy variable Gender where boys = 0 and girls = 1. 

The Parental Education variable is based on the ISCED coding. The highest 
levels of Parental Education were reported for United Arab Emirates (Dubai), 
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Norway, the Russian Federation, Canada (Quebec), Australia, Qatar, Finland, 
and Sweden, and the lowest levels were reported for Honduras, Morocco, 
Botswana, Iran, and Oman. The proportion of girls varied between .52 (Norway, 
Saudi Arabia, and Botswana) and .48 (Morocco, Poland, and Romania). 

The highest means for number of Books at home were observed for 
Sweden, Norway, Australia, Germany, and Finland, while the lowest means 
were observed for Morocco, Honduras, Botswana, Iran, and Azerbaijan. For 
children’s books the highest means were observed for Australia, Sweden, 
Finland, Malta, and Norway and the lowest means were observed for the same 
group of countries as had the smallest number of books at home. The country 
level correlation between the two measures of book availability in the home 
was 0.92. 

The results for the two Activity variables have been computed from the 
IRT-scaled indices in the TIMSS and PIRLS 2011 International Database. The 
highest level of literacy activities was reported by Northern Ireland, the Russian 
Federation, Australia, Ireland, and Croatia, while the highest level of numeracy 
activities was reported by Northern Ireland, Hungary, the Slovak Republic, the 
Czech Republic, and the Russian Federation. There was a general tendency for 
countries that reported a high level of literacy activity to also report a high level 
of numeracy activity, with a correlation of .82. 

For literacy skills the highest means were observed for Singapore, 
Honduras, Hong Kong SAR, and Qatar; for numeracy skills, the highest means 
were observed for Hong Kong SAR, Chinese Taipei, Singapore, and Finland. 
Here, too, there was quite a substantial correlation (.73) between the literacy 
and numeracy measures. 

At the country level, there were negative correlations between the two 
activity variables on the one hand, and the two skills variables on the other hand, 
with correlations ranging between -.26 and -.51. The two measures of number 
of books in the home correlated positively with the two activity measures, and 
negatively with the measures of literacy skills, while there was no correlation 
with the numeracy skills measure. These results suggest that the pattern of 
interrelations among the variables at country level may be quite different from 
the pattern of intercorrelations within countries. 
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Exhibit 4.4: 


Descriptive Statistics 


TIMSS & PIRLS 
2011 


^th 

Grade 


Country 

Parental Education 

Gender 

Books at Home 

Childrens Books 

Mean 

SD 

Mean 

SD 

Mean 

SD 

Mean 

SD 

Australia 

5.16 

0.90 

0.49 

0.50 

3.68 

1.17 

3.91 

1.07 

Austria 

4.55 

0.92 

0.49 

0.50 

3.43 

1.21 

3.33 

1.18 

Azerbaijan 

4.56 

1.11 

0.47 

0.50 

2.14 

1.07 

1.64 

0.85 

Chinese Taipei 

4.58 

1.00 

0.47 

0.50 

2.79 

1.29 

2.94 

1.40 

Croatia 

4.53 

0.88 

0.50 

0.50 

2.74 

1.18 

2.38 

1.10 

Czech Republic 

4.52 

0.88 

0.49 

0.50 

3.44 

1.14 

3.29 

1.05 

Finland 

5.07 

0.96 

0.49 

0.50 

3.53 

1.16 

3.70 

1.04 

Georgia 

4.97 

1.00 

0.48 

0.50 

3.28 

1.32 

2.26 

1.15 

Germany 

4.45 

1.28 

0.49 

0.50 

3.55 

1.22 

3.46 

1.11 

Hong Kong, SAR 

4.00 

1.22 

0.46 

0.50 

2.62 

1.20 

2.68 

1.28 

Hungary 

4.08 

1.39 

0.49 

0.50 

3.43 

1.33 

3.10 

1.27 

Iran, Islamic Republic of 

3.62 

1.35 

0.49 

0.50 

2.03 

1.14 

1.87 

1.08 

Ireland 

4.87 

1.10 

0.49 

0.50 

3.23 

1.26 

3.48 

1.21 

Italy 

4.24 

1.10 

0.50 

0.50 

2.99 

1.26 

2.67 

1.14 

Lithuania 

4.78 

1.09 

0.48 

0.50 

2.91 

1.24 

2.49 

1.14 

Malta 

3.99 

1.22 

0.49 

0.50 

3.21 

1.26 

3.70 

1.05 

Morocco 

3.02 

1.28 

0.48 

0.50 

1.68 

0.97 

1.52 

0.90 

Northern Ireland 

4.51 

1.36 

0.50 

0.50 

3.31 

1.23 

3.68 

1.14 

Norway 

5.37 

0.87 

0.52 

0.50 

3.72 

1.18 

3.68 

1.08 

Oman 

3.96 

1.44 

0.49 

0.50 

2.26 

1.13 

1.71 

0.93 

Poland 

4.31 

1.28 

0.48 

0.50 

3.09 

1.17 

2.95 

1.12 

Portugal 

4.08 

1.39 

0.49 

0.50 

2.84 

1.27 

2.91 

1.22 

Qatar 

5.11 

1.27 

0.47 

0.50 

2.57 

1.27 

2.22 

1.21 

Romania 

3.98 

1.19 

0.48 

0.50 

2.41 

1.31 

2.11 

1.14 

Russian Federation 

5.29 

0.82 

0.49 

0.50 

3.23 

1.16 

2.96 

1.13 

Saudi Arabia 

4.32 

1.50 

0.52 

0.50 

2.28 

1.23 

1.67 

0.96 

Singapore 

4.78 

1.17 

0.49 

0.50 

2.80 

1.21 

3.24 

1.22 

Slovak Republic 

4.51 

0.98 

0.49 

0.50 

3.01 

1.17 

2.74 

1.09 

Slovenia 

4.65 

0.89 

0.48 

0.50 

3.13 

1.15 

3.06 

1.11 

Spain 

4.41 

1.35 

0.49 

0.50 

3.30 

1.23 

3.07 

1.18 

Sweden 

5.05 

0.99 

0.49 

0.50 

3.78 

1.19 

3.76 

1.12 

United Arab Emirates 

5.01 

1.29 

0.50 

0.50 

2.43 

1.23 

2.14 

1.17 

Sixth Grade Countries 

Botswana 

3.31 

1.46 

0.52 

0.50 

1.81 

1.06 

1.57 

0.93 

Honduras 

2.91 

1.40 

0.51 

0.50 

1.76 

1.05 

1.42 

0.84 

Benchmarking Participants 

Quebec, Canada 

5.24 

0.85 

0.50 

0.50 

3.11 

1.21 

3.39 

1.13 

Abu Dhabi, UAE 

4.94 

1.31 

0.50 

0.50 

2.36 

1.21 

2.03 

1.11 

Dubai, UAE 

5.38 

1.07 

0.47 

0.50 

2.71 

1.28 

2.62 

1.31 
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Exhibit 4.4: Descriptive Statistics (Continued) 


TIMSS frPIRLS 
2011 



Literacy Activities 

Numeracy Activities 

Literacy Skills 

Numeracy Skills 


Mean 

SD 

Mean 

SD 

Mean 

SD 

Mean 

SD 

Australia 

10.87 

2.09 

10.66 

1.93 

9.74 

1.72 

9.23 

1.84 

Austria 

10.02 

1.74 

10.45 

1.64 

9.22 

1.87 

9.39 

1.85 

Azerbaijan 

9.50 

1.83 

9.09 

1.81 

9.61 

2.16 

9.43 

2.20 

Chinese Taipei 

8.69 

1.91 

9.18 

2.18 

10.70 

1.48 

11.65 

1.41 

Croatia 

10.75 

1.76 

10.53 

1.70 

10.66 

1.71 

10.37 

1.67 

Czech Republic 

10.29 

1.62 

11.02 

1.53 

9.79 

1.83 

9.97 

1.71 

Finland 

9.78 

1.51 

9.50 

1.43 

10.22 

1.98 

10.64 

1.77 

Georgia 

10.70 

1.93 

9.46 

2.08 

9.64 

2.18 

9.99 

1.92 

Germany 

10.20 

1.72 

10.43 

1.64 

9.23 

1.79 

9.74 

1.77 

Hong Kong, SAR 

8.70 

1.72 

9.15 

1.92 

11.06 

1.57 

11.66 

1.33 

Hungary 

10.29 

1.69 

11.11 

1.70 

8.89 

2.08 

9.69 

1.89 

Iran, Islamic Republic of 

8.91 

1.87 

9.25 

1.97 

9.71 

2.18 

9.40 

2.19 

Ireland 

10.80 

2.02 

10.90 

1.94 

- 

- 

- 

- 

Italy 

10.50 

1.74 

10.30 

1.72 

9.42 

1.74 

9.04 

1.82 

Lithuania 

10.05 

1.72 

9.94 

1.64 

10.19 

1.54 

9.90 

1.86 

Malta 

10.43 

1.94 

10.34 

2.01 

10.36 

1.77 

10.14 

1.78 

Morocco 

8.42 

2.70 

8.19 

2.33 

10.20 

2.29 

9.22 

2.52 

Northern Ireland 

11,19 

2.04 

11.20 

1.89 

9.31 

1.69 

8.59 

1.74 

Norway 

10.08 

1.78 

9.81 

1.61 

9.22 

1.91 

9.49 

1.81 

Oman 

9.25 

1.70 

8.93 

1.88 

10.86 

1.75 

10.47 

1.87 

Poland 

10.41 

1.76 

10.76 

1.64 

10.08 

1.86 

9.67 

1.88 

Portugal 

10.00 

1.88 

9.86 

1.82 

9.46 

1.70 

9.40 

1.79 

Qatar 

9.69 

1.88 

9.76 

2.05 

11.02 

1.79 

10.61 

1.87 

Romania 

9.95 

2.52 

9.74 

2.48 

9.24 

2.15 

9.90 

2.34 

Russian Federation 

11.09 

1.95 

10.90 

1.86 

9.92 

1.94 

10.36 

1.86 

Saudi Arabia 

9.55 

1.84 

9.46 

1.96 

10.76 

2.05 

10.32 

1.94 

Singapore 

9.44 

2.07 

9.70 

2.14 

11.19 

1.60 

11.37 

1.50 

Slovak Republic 

10.52 

1.83 

11.08 

1.78 

8.62 

1.86 

9.32 

2.01 

Slovenia 

10.62 

1.78 

10.41 

1.65 

9.34 

2.01 

9.30 

1.86 

Spain 

10.38 

1.77 

9.96 

1.73 

10.97 

1.81 

10.33 

1.80 

Sweden 

9.99 

1.78 

9.43 

1.69 

10.39 

1.77 

10.24 

1.78 

United Arab Emirates 

9.64 

1.78 

9.91 

1.90 

10.61 

1.82 

10.26 

1.90 

Sixth Grade Countries 

Botswana 

8.68 

2.05 

8.26 

2.19 

10.17 

2.04 

9.05 

2.25 

Honduras 

9.63 

2.24 

8.15 

2.44 

11.13 

1.83 

10.33 

1.94 

Benchmarking Participants 

Quebec, Canada 

10.18 

1.80 

10.38 

1.74 

9.61 

1.71 

9.39 

1.83 

Abu Dhabi, UAE 

9.53 

1.74 

9.79 

1.87 

10.60 

1.85 

10.40 

1.87 

Dubai, UAE 

9.97 

1.86 

10.17 

1.92 

10.64 

1.78 

9.99 

1.91 
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Results from the Common Model for Pooled Data 


The path model for the pooled data was estimated as a two-level model, using 
the procedures described above. The model involved the 34 participating 
countries and 3 benchmarking entities, which represented the between level, 
and 185,475 students. For each variable and each observation, the deviation 
from the international mean was used to compute the pooled within covariance 
matrix, to which the model was fitted. 

As expected, the chi-square test was highly significant, with a mean across 
the five estimations of 1512.86 (df = 78) and a standard deviation of 10.77. 
From a strict statistical point of view this would imply that the model should be 
rejected as not fitting the data. However the mean estimate of RMSEA was 0.01 
with a standard deviation across replications of 0, which indicates excellent fit. 
The CFI estimate was .986, again with a standard deviation of 0. Thus, this index 
also indicates an excellent fit between model and data. Finally, the Standardized 
Root Mean Square Residual (SRMR) for the within level was only 0.012, which 
again indicates a well-fitting model. Thus, we may conclude that the model 
provides an adequate representation of the data. 

The Measurement Model for Pooled Data 

As was described in the Method section, two testlets were built from the items 
included in each of the scales constructed to measure numeracy activities 
(NUMACTl, NUMACT2) and literacy activities (LITACTl, LITACT2), in 
addition to numeracy skills (NUMAB1, NUMAB2) and literacy skills (LITAB1, 
LITAB2) at the start of school. Given that these four scales comprise a relatively 
limited number of items, each of the eight testlets only included between two 
and five items. The small number of items makes the two testlets in each pair less 
than perfectly comparable as indicators of the latent variable. This was apparent 
in the form of some rather large modification indices for the relationships 
between the observed and latent variables. However, no attempt was made to 
adjust for this, for example by moving items from one testlet to another. 

It will be remembered that one general latent Activity variable was 
hypothesized; with positive relationship with NUMACTl, NUMACT2, 
LITACTl and LITACT2, and also that a general latent Ability variable was 
hypothesized, with a positive relationship with NUMAB1, NUMAB2, LITAB1 
and LITAB2. It also was hypothesized that there would be a bipolar NumLitAct 
latent variable, with a negative relationship with NUMACTl and NUMACT2 
and a positive relationship with LITACTl and LITACT2, as well as a bipolar 
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NumLitAb latent variable with a negative relationship with NUMAB1 and 
NUMAB2 and a positive relationship with LITAB1 and LITAB2. When 
estimating the model, the bipolar factors were defined by assigning fixed values 
of - 1 and 1 to the unstandardized factor loadings, while for the two general 
factors one of the indicators was assigned a fixed value of unity and the loadings 
for the other three indicators were freely estimated. The standardized factor 
loadings are easier to interpret, however, so discussion focuses on these (see 
Exhibit 4.5). 


Exhibit 4.5: Standardized Factor Loadings in the Measurement Model for the 

Common Model 
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For the latent variable Activity, all four indicators had large positive loading of 
equal magnitude (0.78). The loadings were smaller for the bipolar NumLitAct 
variable, with absolute values of around .20, meaning that this latent variable 
accounted for only around 4 percent of the observed variance in each testlet. 
Positive values on this bipolar latent variable indicate more literacy than numeracy 
activities, while negative values indicate more numeracy than literacy activities. 

For the latent variable Ability, there also were large positive loadings for 
the four indicators. However, loadings were larger for the two indicators of 
literacy abilities (around .90) than for the two indicators of numeracy abilities 
(around .75). For the latent variable NumLitAb, the bipolar pattern was evident, 
and this latent variable had stronger relationships with the testlets than had the 
bipolar activity factor, and particularly so with respect to the numeracy testlets. 
These patterns of relationships indicate that the literacy skills are of greater 
importance as indicators of a general ability, while the numeracy skills tend to 
be a narrower dimension. Nevertheless, the interpretation of the NumLitAb 
factor is that positive values indicate relatively higher literacy than numeracy 
skills, while negative values indicate relatively higher numeracy skills. 

For the latent variable Books, there were two indicators: number of 
books in the home (NBOOK), and number of children’s books in the home 
(NCBOOK). Both of these indicators had strong and equal relations (.80) to 
the latent variable. 

The Path Model for Pooled Data 

Given the complexity of the full path model and the large number of 
relationships estimated, it was necessary to simplify the presentation of results. 
This was accomplished by presenting the results for Gender and Parental 
Education separately. Because there was no correlation between these two 
variables, this does not cause any loss of information. 

EFFECTS OF PARENTAL EDUCATION The total effects of Parental Education were .33, 
.35, and .35 for mathematics, science, and reading, respectively. While these 
estimates were computed from the direct and indirect effects in the model, we 
could also have computed a correlation between Parental Education and each 
of the three achievement variables to obtain the same results. According to the 
model, the total indirect effects were .12, .12, and .13 for mathematics, science, 
and reading, respectively. The difference between the total effect and the total 
indirect effect is the direct effect. Exhibit 4.6 presents all standardized direct 
effects larger than .05. 
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Exhibit 4.6: Path Diagram for Relations between Parental Education and 

Achievement (All Participants, Pooled Data) 



As may be seen from this Exhibit, the direct effects of Parental Education 
on the three achievement variables agree, within rounding errors, with our 
expectations. These direct effects represent effects of Parental Education that 
the path model cannot account for via mediating variables. It is obvious from 
the model, however, that Books is an important mediating variable, with a 
strong relationship (0.47) between Parental Education and Books, and a direct 
effect of Books on the achievement variables of 0.22, similar to that of Parental 
Education. 

Parental Education also had an indirect effect via the sequence Books, 
Activity, and Ability to achievement. All links in this chain were fairly strong 
and this indirect effect agrees with the theoretical expectations and with findings 
in previous empirical research. Thus, this path is theoretically and empirically 
important and it will be referred to as the Main Path of influence of Parental 
Education on achievement. There also was another important path, overlapping 
the Main Path to a great extent, which went directly from Parental Education 
to Activity, circumventing Books. It should also be pointed out that there was 
no direct effect of Activity on achievement in the Main Path, the entire effect 
being mediated via Ability. 
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In the Common model there was no direct effect of Parental Education on 
NumLitAct, and only a very weak indirect effect via Books, which was negative. 
Thus, NumLitAct did not mediate effects of Parental Education on achievement. 

There was, however, a pattern of indirect effects of NumLitAct on the 
three achievement variables that went via Ability. There also was a negative 
direct effect of NumLitAct on Mathematics achievement. These results mean 
that homes which reported a stronger emphasis on literacy activities than on 
numeracy activities also reported a higher level of Ability, which in turn had a 
positive direct effect on achievement in all three domains. This is an interesting 
result, and one possible interpretation is that emphasis on literacy activities 
has a positive effect on development of both literacy and numeracy skills. A 
partially different interpretation is that numeracy skills at the beginning of 
primary school tend to involve both reading and writing, because expression 
of numeracy skills often requires use of literacy skills. 

It may seem strange that there was a negative direct effect of NumLitAct 
on mathematics achievement. However, there was a positive indirect effect of 
NumLitAct on mathematics achievement, which was mediated via Ability. This 
positive indirect effect of NumLitAct on mathematics achievement thus partially 
balances out the negative direct effect of NumLitAct. Because there was no 
negative direct effect of NumLitAct on science or reading achievement, the net 
effect is that the emphasis on literacy activity will cause a profile of achievement 
with a relative strength in reading and science compared to mathematics. 

Thus, even though NumLitAct did not mediate effects of Parental 
Education on achievement in the current model, the NumLitAct variable does 
seem to be involved in interesting patterns of relations. 

EFFECTS OF GENDER The total effects of Gender were .00, .02 and .12 on 
mathematics, science, and reading, respectively. These results imply that in 
the pooled data there was essentially no gender difference in mathematics or 
science, but a rather substantial Gender difference in favor of girls with respect 
to reading achievement. The total indirect effects of Gender were .01, .01 and 
.02 on mathematics, science, and reading, respectively, so only a small part of 
the Gender effect was mediated via the variables in the model. 

Exhibit 4.7 presents the path diagram in which Gender is the independent 
variable in focus. 
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Exhibit 4.7: Path Diagram for Relations between Gender and Achievement 

(All Participants, Pooled Data) 



As may be expected from the total and indirect effects, there was a 
direct effect of .10 on reading. There was no direct effect of Gender on Books 
or Activity so the Main Path did not mediate any of the effect of Gender on 
achievement. 

There was an effect of Gender on NumLitAct, which implies that for 
girls activities in the home tended to be more oriented towards literacy than 
numeracy. NumLitAct in turn influenced Ability, which had direct effects on 
all three fourth grade student achievement variables, so there were indirect 
effects of NumLitAct on achievement. There also was a weaker direct effect of 
Gender on NumLitAb, as well as an indirect effect via NumLitAct. However, 
because there was no direct effect of NumLitAb on any of the three achievement 
variables, NumLitAb did not mediate much of the total Gender effect. 

Discussion of Results from the Common Model for Pooled Data 
The analyses of indirect effects of Parental Education on achievement provide 
strong support for the hypothesized chain of influence via Books, Activity, 
and Ability to achievement. While this Main Path is important theoretically 
and empirically it may be noted that important indirect effects went through 
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other paths. Thus, Books was an important variable through which Parental 
Education exerted influence, and was not only part of the Main Path but also 
had substantial direct effects on the three achievement variables. These direct 
effects maybe assumed to be mediated via variables not included in the model, 
such as parental expectations, and parents’ function as role models with respect 
to reading activities. 

It was expected that NumLitAct would affect NumLitAb, which it did, 
albeit to a limited extent. However, NumLitAct was more strongly related to 
Ability, and because Ability had effects on achievement, the mediating effect of 
NumLitAct on achievement went via Ability, causing similar effects on all three 
domains of achievement. There was, however, some differential effect because 
of the negative direct effect of NumLitAct on mathematics achievement. 

Overall Description of Results from Country by 
Country Analyses 

The Common model discussed in the previous section provides a synopsis of 
the general pattern of relationships among the variables, but it does not give 
any information about differences in the pattern of relationships among the 
variables across countries. In order to investigate such differences, we have fitted 
the path model separately to the data for each of the 37 participants. This section 
provides an overview of the results, while the next section presents the models 
country by country. 

Total, Direct, and Total Indirect Effects of Parental Education and 
Gender on Achievement 

This section reports the pattern of outcomes across countries with respect to 
total, direct, and total indirect effects. 

TOTAL EFFECTS OF PARENTAL EDUCATION AND GENDER ON ACHIEVEMENT Exhibit 4.8 
presents estimates of the total effects of Parental Education and Gender on the 
three fourth grade student achievement variables. 

Parental Education had an average effect on achievement across countries 
of around .34. These estimates agree almost perfectly with those obtained in the 
analysis of the pooled data. But here we can see that there was a considerable 
variation across countries, and also across the three achievement domains. 

For Hungary, Iran, Romania, Poland, and Botswana Parental Education 
had total effects which exceeded .40 in all three domains. The lowest impact of 
Parental Education was observed for Azerbaijan and Hong Kong SAR, where 
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Exhibit 4.8: Estimates of Total Standardized Effects 


Country 


Total Effect of Parental Education 



Total Effect of Gender 


Mathematics 

Science 

Reading 

Mathematics 

Science 

Reading 

Australia 

0.333 (0.024) 

0.351 (0.025) 

0.330 (0.023) 

-0.025 (0.020) 

-0.001 (0.021) 

0.122 (0.019) 

Austria 

0.307 (0.019) 

0.334 (0.021) 

0.317 (0.019) 

-0.069 (0.019) 

-0.086 (0.019) 

0.063 (0.018) 

Azerbaijan 

0.109 (0.024) 

0.137 (0.024) 

0.148 (0.023) 

0.036 (0.023) 

0.032 (0.023) 

0.099 (0.019) 

Chinese Taipei 

0.370 (0.022) 

0.387 (0.019) 

0.335 (0.019) 

0.014 (0.018) 

-0.046 (0.019) 

0.100 (0.016) 

Croatia 

0.309 (0.023) 

0.318 (0.024) 

0.305 (0.022) 

-0.084 (0.021) 

-0.038 (0.019) 

0.123 (0.022) 

Czech Republic 

0.306 (0.024) 

0.289 (0.023) 

0.285 (0.024) 

-0.08 (0.019) 

-0.094 (0.021) 

0.061 (0.019) 

Finland 

0.289 (0.023) 

0.280 (0.023) 

0.275 (0.020) 

-0.062 (0.021) 

-0.012 (0.019) 

0.166 (0.018) 

Georgia 

0.282 (0.024) 

0.289 (0.022) 

0.313 (0.021) 

0.044 (0.020) 

0.057 (0.023) 

0.153 (0.019) 

Germany 

0.359 (0.020) 

0.380 (0.019) 

0.361 (0.018) 

-0.068 (0.017) 

-0.088 (0.025) 

0.064 (0.018) 

Hong Kong SAR 

0.160 (0.028) 

0.150 (0.027) 

0.116 (0.026) 

-0.054 (0.021) 

-0.069 (0.020) 

0.131 (0.019) 

Hungary 

0.549 (0.020) 

0.550 (0.021) 

0.530 (0.021) 

-0.026 (0.018) 

-0.03 (0.019) 

0.099 (0.015) 

Iran, Islamic Rep. of 

0.441 (0.025) 

0.446 (0.024) 

0.433 (0.023) 

-0.023 (0.029) 

-0.034 (0.028) 

0.112 (0.026) 

Ireland 

0.334 (0.020) 

0.341 (0.021) 

0.343 (0.023) 

-0.02 (0.027) 

-0.001 (0.028) 

0.113 (0.022) 

Italy 

0.238 (0.025) 

0.275 (0.022) 

0.298 (0.021) 

-0.063 (0.022) 

-0.053 (0.018) 

0.033 (0.018) 

Lithuania 

0.356 (0.022) 

0.352 (0.022) 

0.345 (0.023) 

-0.007 (0.019) 

-0.008 (0.019) 

0.148 (0.019) 

Malta 

0.339 (0.023) 

0.449 (0.022) 

0.444 (0.022) 

-0.041 (0.022) 

-0.037 (0.023) 

0.095 (0.024) 

Morocco 

0.185 (0.036) 

0.193 (0.032) 

0.241 (0.033) 

0.029 (0.019) 

0.040 (0.019) 

0.132 (0.018) 

Northern Ireland 

0.378 (0.029) 

0.387 (0.029) 

0.361 (0.028) 

0.016 (0.023) 

0.018 (0.025) 

0.130 (0.021) 

Norway 

0.253 (0.027) 

0.278 (0.026) 

0.263 (0.023) 

-0.04 (0.022) 

-0.006 (0.024) 

0.133 (0.023) 

Oman 

0.304 (0.023) 

0.305 (0.024) 

0.319 (0.024) 

0.131 (0.016) 

0.138 (0.017) 

0.204 (0.014) 

Poland 

0.427 (0.018) 

0.441 (0.017) 

0.431 (0.016) 

-0.059 (0.024) 

-0.024 (0.018) 

0.109 (0.019) 

Portugal 

0.303 (0.031) 

0.298 (0.029) 

0.314 (0.023) 

-0.042 (0.022) 

-0.032 (0.021) 

0.111 (0.017) 

Qatar 

0.394 (0.025) 

0.383 (0.028) 

0.395 (0.024) 

0.061 (0.030) 

0.105 (0.033) 

0.144 (0.027) 

Romania 

0.430 (0.035) 

0.466 (0.031) 

0.490 (0.028) 

-0.017 (0.018) 

-0.006 (0.018) 

0.079 (0.018) 

Russian Federation 

0.265 (0.026) 

0.269 (0.024) 

0.298 (0.022) 

0.008 (0.018) 

-0.008 (0.020) 

0.137 (0.017) 

Saudi Arabia 

0.176 (0.032) 

0.248 (0.029) 

0.243 (0.028) 

0.064 (0.051) 

0.199 (0.046) 

0.274 (0.041) 

Singapore 

0.393 (0.019) 

0.437 (0.017) 

0.408 (0.017) 

0.020 (0.017) 

-0.028 (0.017) 

0.101 (0.016) 

Slovak Republic 

0.371 (0.028) 

0.375 (0.028) 

0.376 (0.024) 

-0.045 (0.015) 

-0.046 (0.016) 

0.079 (0.018) 

Slovenia 

0.376 (0.019) 

0.386 (0.024) 

0.347 (0.020) 

-0.061 (0.022) 

-0.016 (0.024) 

0.123 (0.021) 

Spain 

0.373 (0.022) 

0.333 (0.027) 

0.314 (0.027) 

-0.082 (0.019) 

-0.071 (0.019) 

0.034 (0.018) 

Sweden 

0.324 (0.025) 

0.340 (0.026) 

0.339 (0.025) 

-0.046 (0.019) 

-0.027 (0.021) 

0.108 (0.020) 

United Arab Emirates 

0.386 (0.017) 

0.402 (0.016) 

0.415 (0.015) 

0.040 (0.023) 

0.090 (0.023) 

0.136 (0.023) 

Sixth Grade Countries 

Botswana 

0.405 (0.037) 

0.445 (0.035) 

0.478 (0.035) 

0.101 (0.019) 

0.061 (0.019) 

0.148 (0.018) 

Honduras 

0.343 (0.058) 

0.355 (0.054) 

0.338 (0.056) 

-0.08 (0.023) 

-0.064 (0.026) 

0.071 (0.025) 

Benchmarking Participants 

Quebec, Canada 

0.252 (0.026) 

0.293 (0.025) 

0.273 (0.023) 

-0.082 (0.021) 

-0.061 (0.023) 

0.107 (0.018) 

Abu Dhabi, UAE 

0.399 (0.024) 

0.392 (0.024) 

0.397 (0.024) 

0.077 (0.032) 

0.144 (0.030) 

0.185 (0.028) 

Dubai, UAE 

0.405 (0.025) 

0.420 (0.025) 

0.421 (0.025) 

-0.01 (0.035) 

0.020 (0.037) 

0.071 (0.039) 


International Avg. 0.326 (0.024) 0.340 (0.024) 0.335 (0.023) 0.042 (0.022) 0.085 (0.022) 0.116 (0.020) 

( ) Standard errors appear in parentheses. 
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effects were lower than .16 in all three achievement domains. Thus, there 
were considerable differences in the amount of relationship between Parental 
Education and achievement across countries, even though it also may be noted 
that for many countries effects were between .30 and .40. From the list of 
countries with high and low impact it is not possible to determine any simple 
and clear grouping of countries which may explain the differences. Among 
participants with high impact, some were East European countries. However, 
the Russian Federation was among the countries with lowest impact, thus the 
pattern is far from clear. Among East Asian countries there were both examples 
of countries with the highest impact (Singapore) and the lowest impact (Hong 
Kong SAR). Similarly among developing countries, there were examples of high 
impact of Parental Education (Botswana) and low impact (e.g., Morocco). These 
examples indicate that the amount of effect of Parental Education on educational 
achievement cannot be accounted for in simple terms. 

In addition, results presented in Exhibit 4.8 indicate that there was no 
significant average effect of Gender on achievement in mathematics or science, 
while there was an average effect of .12 on reading. This average agrees with the 
estimate obtained in the analysis of the pooled data, and it agrees with previous 
findings of consistent differences in favor of girls on reading literacy. 
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However, even though there were no overall average gender differences 
in mathematics and science, there were countries were either boys or girls 
excelled. Significant differences in mathematics achievement in favor of girls 
were observed for five participants: Oman, Botswana, the Emirate of Abu 
Dhabi, Saudi Arabia, and Qatar. Among these participants, there also were 
significant differences for girls in science achievement. This same group of 
participants, with a few exceptions, also had considerable differences in reading 
achievement, with standardized coefficients as high as around .20. Finland also 
had a considerable advantage for girls in reading achievement. 

For about a dozen participants, there were significant differences in favor 
of boys in mathematics, but in no case larger than 0.08 (e.g., Croatia, the 
Canadian province of Quebec, Spain, the Czech Republic, Hungary, Austria, 
and Germany). Most of these countries had a similar pattern of differences in 
science achievement. For only two countries (Spain and Italy) a non-significant 
gender difference was observed for reading achievement. For the countries with 
small differences in favor of girls for reading, there tended to be a significant 
advantage for boys in mathematics and science. 


TIMSS & PIRLS 

International Study Center 

Lynch School of Education, Boston College 


EFFECTS OF HOME BACKGROUND ON STUDENT ACHIEVEMENT IN 
READING, MATHEMATICS, AND SCIENCE AT THE FOURTH GRADE 

CHAPTER 4 


215 



DIRECT EFFECTS OF PARENTAL EDUCATION AND GENDER ON ACHIEVEMENT The total 
effects presented above arise from direct effects and indirect effects. In order 
to understand the composition of the total effect in the different countries, it 
is useful to examine these two sources of effects separately. This section first 
discusses the direct effects, and then investigates the indirect effects. Exhibit 
4.9 presents the direct effects of Parental Education on the other variables in 
the path model. 

The mean standardized regression coefficient (( 3 ) for the direct effect 
of Parental Education on mathematics achievement was 0.19 (sd = 0.07), as 
compared to 0.33 for the total effect. Countries with the smallest direct effects 
of Parental Education on mathematics achievement ((3 < .13) were Azerbaijan, 
Hong Kong SAR, Saudi Arabia, Austria, Portugal, and Sweden. Countries with 
the largest direct effects of Parental Education on mathematics achievement 
((3 > .27) included Botswana, Honduras, Qatar, Poland, and Hungary. There was 
a very high level of agreement between the pattern of results for mathematics 
and science achievement; the correlation between the parameter estimates for 
mathematics and science was 0.92. 

The mean direct effect of Parental Education on reading achievement was 
about the same as for mathematics and science (mean = 0.20, sd = 0.07). The 
correlation between the parameter estimates for reading and mathematics was 
0.89, while the correlation was 0.97 for reading and science. Thus, there was more 
agreement in the pattern of outcomes for reading and science than for reading 
and mathematics. For reading, particularly small direct effects of Parental 
Education were observed for Hong Kong SAR and for the Nordic countries. 

The mean effect of Gender on mathematics achievement was small 
(mean = -0.03, sd = 0.05). However, in about half of the countries, there were 
significant direct effects in favor of boys. The countries with the largest direct 
effects included Slovenia, Sweden, Croatia, Spain, and the Czech Republic. For 
two countries (Botswana and Oman) there was a significant direct effect in 
favor of girls. 

For science, too, the mean direct effect of Gender was small 
(mean = -0.02, sd = 0.07), although the pattern of direct effects of gender 
differed across countries. For about half of the countries, there was a significant 
direct effect in favor of boys, and was largest in Hong Kong SAR, Sweden, the 
Czech Republic, the Canadian province of Quebec, Spain, and Germany. For 
seven countries, there was a significant direct effect in favor of girls, and was 
largest in Saudi Arabia, Oman, the Emirate of Abu Dhabi, Qatar, and United 
Arab Emirates. 


TIMSS AND PIRLS 201 1 RELATIONSHIPS REPORT 

216 CHAPTER 4 



TIMSS & PIRLS 

International Study Center 

Lynch School of Education, Boston College 



Exhibit 4.9: Standardized Direct Effects of Parental Education and 

Gender on Achievement, Grade 4 


TIMSS &PIRLS 
2011 


Country 


Parental Education 



Gender 


Mathematics 

Science 

Reading 

Mathematics 

Science 

Reading 

Australia 

0.225 (0.027) 

0.216 (0.028) 

0.212 (0.026) 

-0.021 (0.020) 

-0.006 (0.021) 

0.115 (0.019) 

Austria 

0.092 (0.021) 

0.100 (0.023) 

0.095 (0.021) 

-0.046 (0.021) 

-0.082 (0.018) 

0.051 (0.017) 

Azerbaijan 

0.060 (0.028) 

0.078 (0.027) 

0.114 (0.026) 

0.031 (0.025) 

0.032 (0.025) 

0.100 (0.021) 

Chinese Taipei 

0.200 (0.022) 

0.220 (0.021) 

0.200 (0.021) 

-0.004 (0.018) 

-0.068 (0.018) 

0.068 (0.017) 

Croatia 

0.173 (0.024) 

0.169 (0.028) 

0.161 (0.025) 

-0.095 (0.022) 

-0.054 (0.020) 

0.085 (0.023) 

Czech Republic 

0.166 (0.023) 

0.129 (0.025) 

0.129 (0.022) 

-0.085 (0.018) 

-0.1 (0.021) 

0.048 (0.019) 

Finland 

0.160 (0.024) 

0.130 (0.027) 

0.117 (0.022) 

-0.076 (0.020) 

-0.037 (0.020) 

0.108 (0.020) 

Georgia 

0.148 (0.024) 

0.146 (0.023) 

0.174 (0.022) 

0.042 (0.021) 

0.051 (0.023) 

0.140 (0.020) 

Germany 

0.169 (0.022) 

0.150 (0.026) 

0.152 (0.023) 

-0.048 (0.019) 

-0.086 (0.026) 

0.052 (0.020) 

Hong Kong SAR 

0.072 (0.025) 

0.051 (0.029) 

0.038 (0.025) 

-0.084 (0.020) 

-0.11 (0.020) 

0.091 (0.020) 

Hungary 

0.272 (0.027) 

0.264 (0.025) 

0.288 (0.024) 

-0.006 (0.016) 

-0.018 (0.017) 

0.103 (0.014) 

Iran, Islamic Rep. of 

0.248 (0.025) 

0.251 (0.026) 

0.251 (0.024) 

-0.013 (0.029) 

-0.023 (0.027) 

0.122 (0.025) 

Ireland 

0.165 (0.023) 

0.156 (0.025) 

0.158 (0.027) 

-0.037 (0.027) 

-0.014 (0.028) 

0.089 (0.021) 

Italy 

0.146 (0.025) 

0.139 (0.023) 

0.163 (0.024) 

-0.059 (0.023) 

-0.064 (0.020) 

0.009 (0.020) 

Lithuania 

0.187 (0.020) 

0.175 (0.022) 

0.159 (0.022) 

-0.044 (0.021) 

-0.063 (0.022) 

0.077 (0.019) 

Malta 

0.208 (0.027) 

0.303 (0.026) 

0.299 (0.027) 

-0.062 (0.022) 

-0.058 (0.020) 

0.073 (0.021) 

Morocco 

0.181 (0.027) 

0.162 (0.031) 

0.191 (0.027) 

0.023 (0.017) 

0.030 (0.019) 

0.118 (0.017) 

Northern Ireland 

0.247 (0.036) 

0.225 (0.033) 

0.219 (0.033) 

-0.014 (0.026) 

-0.017 (0.029) 

0.090 (0.025) 

Norway 

0.155 (0.033) 

0.109 (0.027) 

0.107 (0.025) 

-0.04 (0.025) 

-0.04 (0.025) 

0.078 (0.023) 

Oman 

0.214 (0.023) 

0.222 (0.023) 

0.225 (0.023) 

0.107 (0.016) 

0.114 (0.017) 

0.178 (0.014) 

Poland 

0.273 (0.023) 

0.282 (0.019) 

0.288 (0.020) 

-0.077 (0.026) 

-0.06 (0.020) 

0.070 (0.018) 

Portugal 

0.127 (0.039) 

0.128 (0.040) 

0.138 (0.036) 

-0.029 (0.023) 

-0.028 (0.022) 

0.109 (0.020) 

Qatar 

0.307 (0.026) 

0.302 (0.028) 

0.316 (0.026) 

0.047 (0.028) 

0.084 (0.030) 

0.122 (0.025) 

Romania 

0.213 (0.034) 

0.227 (0.033) 

0.251 (0.032) 

-0.031 (0.018) 

-0.02 (0.018) 

0.063 (0.018) 

Russian Federation 

0.154 (0.025) 

0.143 (0.024) 

0.166 (0.021) 

-0.013 (0.019) 

-0.029 (0.019) 

0.108 (0.017) 

Saudi Arabia 

0.086 (0.041) 

0.151 (0.037) 

0.159 (0.030) 

0.025 (0.051) 

0.164 (0.048) 

0.230 (0.042) 

Singapore 

0.247 (0.017) 

0.267 (0.016) 

0.243 (0.015) 

-0.01 (0.015) 

-0.065 (0.015) 

0.061 (0.014) 

Slovak Republic 

0.161 (0.027) 

0.162 (0.026) 

0.162 (0.023) 

-0.046 (0.015) 

-0.059 (0.017) 

0.067 (0.017) 

Slovenia 

0.228 (0.021) 

0.213 (0.027) 

0.176 (0.020) 

-0.1 (0.022) 

-0.065 (0.024) 

0.069 (0.022) 

Spain 

0.206 (0.027) 

0.143 (0.028) 

0.155 (0.029) 

-0.089 (0.023) 

-0.088 (0.018) 

0.003 (0.018) 

Sweden 

0.128 (0.030) 

0.095 (0.028) 

0.119 (0.029) 

-0.096 (0.021) 

-0.103 (0.022) 

0.009 (0.020) 

United Arab Emirates 

0.248 (0.016) 

0.269 (0.015) 

0.261 (0.015) 

0.028 (0.022) 

0.074 (0.022) 

0.118 (0.021) 

Sixth Grade Countries 

Botswana 

0.316 (0.034) 

0.341 (0.031) 

0.375 (0.031) 

0.085 (0.020) 

0.046 (0.018) 

0.138 (0.018) 

Honduras 

0.312 (0.054) 

0.307 (0.050) 

0.291 (0.052) 

-0.082 (0.024) 

-0.062 (0.026) 

0.072 (0.025) 

Benchmarking Participants 

Quebec, Canada 

0.175 (0.027) 

0.183 (0.027) 

0.170 (0.025) 

-0.075 (0.022) 

-0.095 (0.027) 

0.055 (0.021) 

Abu Dhabi, UAE 

0.267 (0.024) 

0.270 (0.023) 

0.251 (0.023) 

0.052 (0.030) 

0.113 (0.028) 

0.152 (0.026) 

Dubai, UAE 

0.247 (0.024) 

0.259 (0.024) 

0.252 (0.024) 

-0.039 (0.031) 

-0.015 (0.032) 

0.032 (0.034) 


International Avg. 

0.183 (0.026) 

0.181 (0.026) 

0.184 (0.025) 

0.043 (0.022) 

0.078 (0.022) 

0.088 (0.020) 

International Std. Dev. 

0.060 (0.006) 

0.067 (0.005) 

0.066 (0.005) 

0.029 (0.006) 

0.048 (0.006) 

0.046 (0.005) 


( ) Standard errors appear in parentheses. 
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The direct effect of Gender on reading achievement was positive 
(mean = 0.09, sd = 0.05). In no country was there a direct effect in favor of boys; 
however, for four countries (Spain, Italy, Sweden, and the Emirate of Dubai) 
there was no significant effect. For all other countries, there was a significant 
direct effect of Gender in favor of girls. 

TOTAL INDIRECT EFFECTS OF PARENTAL EDUCATION AND GENDER ON ACHIEVEMENT The total 
indirect effect is due to the sum of all the indirect effects, and Exhibit 4.10 
presents the total indirect effects for Parental Education and Gender. This 
Exhibit also presents the percentage that the total indirect effect amounts to of 
the total effect. It should be observed, however, that for Gender the percentages 
are only presented for reading, because the total effect was in many cases close 
to zero or negative for Gender with respect to mathematics and science. 

The average of the total indirect effect of Parental Education on mathematics 
was 0.14 (sd = 0.06). On average internationally, 41 percent of the total effect was 
indirect (sd = 13). The largest proportions of indirect effects were observed for 
Austria, Sweden, Portugal, the Slovak Republic, and Hong Kong, where in all 
cases 55 percent or more of the total effect was indirect. The relatively smallest 
indirect effects were observed for Morocco, Honduras, Botswana, and Qatar, 
where in all cases less than 22 percent of the total effect was indirect. 
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For science, the average total indirect effect of Parental Education was 0.15 
(sd = 0.06). On average internationally, 45 percent of the total effect was indirect 
(sd = 14). The proportions of the total effects accounted for by the indirect 
effects where similar to those observed for mathematics. For reading, the pattern 
of results was highly similar to the pattern observed for science. 

For the effects of Gender on mathematics and science, as mentioned above, 
generally it is not meaningful to compute the percentage of indirect effects out 
of the total effect, because the latter in many cases was close to zero. There were, 
however, significant indirect effects of Gender on both mathematics and science 
achievement for eleven participants: Hong Kong SAR, Fithuania, Oman, Saudi 
Arabia, Singapore, Slovenia, Sweden, United Arab Emirates, the Emirates of 
Dubai and Abu Dhabi, and Northern Ireland. 

For a majority of the countries, the indirect effect of Gender on reading 
achievement was significant. The average total indirect effect was 0.03 (sd = 
0.02), which indicates considerable variation across countries. The indirect effect 
comprised more than 90 percent of the total effect for Sweden and Spain, and it 
comprised more than 40 percent for Italy, the Emirate of Dubai, Fithuania, the 
Canadian province of Quebec, Slovenia, Norway, and Singapore. The indirect 
effect was close to zero for Iran, Hungary, Honduras, Azerbaijan, Portugal, 
Australia, Botswana, and Georgia. 


TIMSS & PIRLS 

International Study Center 

Lynch School of Education, Boston College 


EFFECTS OF HOME BACKGROUND ON STUDENT ACHIEVEMENT IN 
READING, MATHEMATICS, AND SCIENCE AT THE FOURTH GRADE 

CHAPTER 4 


219 



Exhibit 4.10: 


Total Indirect Standardized Effects 


TIMSS fr PIRLS 
2011 


^th 

Grade 




T 

otal Indirect Effect of Parental Education 


Country 

Mathematics 

Science 


Reading 



Effect 

% of Total 

Effect 

% of Total 

Effect 

% of Total 

Australia 

0.107 (0.014) 

32 

0.135 (0.014) 

38 

0.117 (0.015) 

35 

Austria 

0.216 (0.016) 

70 

0.234 (0.015) 

70 

0.223 (0.013) 

70 

Azerbaijan 

0.049 (0.015) 

45 

0.060 (0.015) 

44 

0.034 (0.014) 

23 

Chinese Taipei 

0.170 (0.013) 

46 

0.167 (0.014) 

43 

0.134 (0.013) 

40 

Croatia 

0.136 (0.016) 

44 

0.149 (0.016) 

47 

0.144 (0.015) 

47 

Czech Republic 

0.140 (0.013) 

46 

0.160 (0.015) 

55 

0.156 (0.014) 

55 

Finland 

0.129 (0.020) 

45 

0.150 (0.016) 

54 

0.158 (0.013) 

57 

Georgia 

0.134 (0.022) 

48 

0.143 (0.022) 

49 

0.138 (0.019) 

44 

Germany 

0.189 (0.017) 

53 

0.230 (0.020) 

61 

0.210 (0.016) 

58 

Hong Kong SAR 

0.088 (0.019) 

55 

0.100 (0.019) 

67 

0.078 (0.017) 

67 

Hungary 

0.277 (0.025) 

50 

0.286 (0.023) 

52 

0.242 (0.023) 

46 

Iran, Islamic Rep. of 

0.193 (0.025) 

44 

0.195 (0.024) 

44 

0.182 (0.023) 

42 

Ireland 

0.169 (0.015) 

51 

0.185 (0.018) 

54 

0.185 (0.016) 

54 

Italy 

0.091 (0.014) 

38 

0.136 (0.016) 

49 

0.135 (0.013) 

45 

Lithuania 

0.170 (0.016) 

48 

0.177 (0.016) 

50 

0.186 (0.015) 

54 

Malta 

0.131 (0.018) 

39 

0.146 (0.016) 

33 

0.145 (0.017) 

33 

Morocco 

0.004 (0.022) 

2 

0.031 (0.020) 

16 

0.050 (0.019) 

21 

Northern Ireland 

0.131 (0.023) 

35 

0.162 (0.025) 

42 

0.142 (0.023) 

39 

Norway 

0.098 (0.020) 

39 

0.169 (0.020) 

61 

0.156 (0.020) 

59 

Oman 

0.090 (0.008) 

30 

0.083 (0.009) 

27 

0.094 (0.009) 

29 

Poland 

0.154 (0.016) 

36 

0.159 (0.014) 

36 

0.143 (0.015) 

33 

Portugal 

0.177 (0.028) 

58 

0.170 (0.029) 

57 

0.176 (0.028) 

56 

Qatar 

0.087 (0.018) 

22 

0.081 (0.016) 

21 

0.079 (0.015) 

20 

Romania 

0.216 (0.026) 

50 

0.238 (0.024) 

51 

0.240 (0.023) 

49 

Russian Federation 

0.111 (0.014) 

42 

0.125 (0.014) 

46 

0.132 (0.013) 

44 

Saudi Arabia 

0.090 (0.025) 

51 

0.097 (0.023) 

39 

0.085 (0.023) 

35 

Singapore 

0.146 (0.012) 

37 

0.170 (0.012) 

39 

0.165 (0.012) 

40 

Slovak Republic 

0.209 (0.020) 

56 

0.214 (0.019) 

57 

0.214 (0.019) 

57 

Slovenia 

0.148 (0.016) 

39 

0.173 (0.016) 

45 

0.171 (0.016) 

49 

Spain 

0.167 (0.019) 

45 

0.190 (0.020) 

57 

0.160 (0.019) 

51 

Sweden 

0.196 (0.018) 

60 

0.245 (0.017) 

72 

0.220 (0.017) 

65 

United Arab Emirates 

0.137 (0.011) 

35 

0.134 (0.011) 

33 

0.154 (0.011) 

37 

Sixth Grade Countries 

Botswana 

0.089 (0.018) 

22 

0.104 (0.016) 

23 

0.103 (0.016) 

22 

Honduras 

0.031 (0.017) 

9 

0.048 (0.017) 

14 

0.047 (0.017) 

14 

Benchmarking Participants 

Quebec, Canada 

0.078 (0.015) 

31 

0.111 (0.014) 

38 

0.103 (0.014) 

38 

Abu Dhabi, UAE 

0.132 (0.020) 

33 

0.122 (0.021) 

31 

0.146 (0.022) 

37 

Dubai, UAE 

0.158 (0.017) 

39 

0.161 (0.016) 

38 

0.169 (0.016) 

40 


International Avg. 

0.142 (0.018) 

43 

0.159 (0.018) 

47 

0.152 (0.017) 

46 

International Std. Dev. 

0.054 (0.005) 

12 

0.056 (0.004) 

13 

0.052 (0.004) 

13 


( ) Standard errors appear in parentheses. 
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Exhibit 4.10: Total Indirect Standardized Effects (Continued) 


Total Indirect Effect of Gender 

Country 

Mathematics 

Science 

Reading 



Effect 

Effect 

Effect 

% of Total 

Australia 

-0.003 (0.012) 

0.005 (0.012) 

0.007 (0.011) 

6 

Austria 

-0.023 (0.014) 

-0.005 (0.012) 

0.012 (0.013) 

19 

Azerbaijan 

0.005 (0.013) 

0.000 (0.013) 

0.000 (0.012) 

0 

Chinese Taipei 

0.017 (0.010) 

0.022 (0.009) 

0.032 (0.008) 

32 

Croatia 

0.011 (0.010) 

0.016 (0.008) 

0.038 (0.009) 

31 

Czech Republic 

0.005 (0.010) 

0.006 (0.010) 

0.012 (0.009) 

20 

Finland 

0.014 (0.014) 

0.025 (0.013) 

0.059 (0.013) 

36 

Georgia 

0.002 (0.008) 

0.006 (0.008) 

0.012 (0.007) 

8 

Germany 

-0.019 (0.015) 

-0.001 (0.014) 

0.012 (0.013) 

19 

Hong Kong SAR 

0.029 (0.008) 

0.040 (0.008) 

0.040 (0.008) 

31 

Hungary 

-0.02 (0.008) 

-0.012 (0.008) 

-0.004 (0.007) 

-4 

Iran, Islamic Rep. of 

-0.01 (0.012) 

-0.011 (0.012) 

-0.01 (0.012) 

-9 

Ireland 

0.018 (0.009) 

0.013 (0.011) 

0.024 (0.010) 

21 

Italy 

-0.004 (0.009) 

0.011 (0.010) 

0.024 (0.010) 

73 

Lithuania 

0.037 (0.014) 

0.055 (0.014) 

0.071 (0.012) 

48 

Malta 

0.021 (0.008) 

0.021 (0.009) 

0.022 (0.009) 

23 

Morocco 

0.006 (0.009) 

0.009 (0.008) 

0.014 (0.007) 

11 

Northern Ireland 

0.031 (0.014) 

0.036 (0.015) 

0.039 (0.013) 

30 

Norway 

-0.001 (0.017) 

0.034 (0.016) 

0.056 (0.015) 

42 

Oman 

0.024 (0.006) 

0.024 (0.006) 

0.025 (0.006) 

12 

Poland 

0.018 (0.010) 

0.037 (0.011) 

0.039 (0.009) 

36 

Portugal 

-0.014 (0.010) 

-0.004 (0.009) 

0.001 (0.010) 

1 

Qatar 

0.014 (0.009) 

0.021 (0.008) 

0.022 (0.007) 

15 

Romania 

0.014 (0.007) 

0.014 (0.007) 

0.016 (0.007) 

20 

Russian Federation 

0.021 (0.007) 

0.021 (0.008) 

0.030 (0.007) 

22 

Saudi Arabia 

0.039 (0.014) 

0.035 (0.014) 

0.044 (0.014) 

16 

Singapore 

0.029 (0.007) 

0.038 (0.008) 

0.040 (0.007) 

40 

Slovak Republic 

0.002 (0.009) 

0.013 (0.009) 

0.012 (0.009) 

15 

Slovenia 

0.039 (0.011) 

0.049 (0.010) 

0.055 (0.009) 

45 

Spain 

0.008 (0.012) 

0.017 (0.012) 

0.031 (0.010) 

91 

Sweden 

0.050 (0.015) 

0.076 (0.016) 

0.099 (0.016) 

92 

United Arab Emirates 

0.012 (0.005) 

0.016 (0.006) 

0.018 (0.006) 

13 

Sixth Grade Countries 

Botswana 

0.016 (0.007) 

0.015 (0.007) 

0.010 (0.007) 

7 

Honduras 

0.001 (0.007) 

-0.003 (0.008) 

-0.001 (0.007) 

-1 

Benchmarking Participants 

Quebec, Canada 

-0.007 (0.014) 

0.034 (0.015) 

0.051 (0.016) 

48 

Abu Dhabi, UAE 

0.026 (0.008) 

0.031 (0.008) 

0.033 (0.009) 

18 

Dubai, UAE 

0.029 (0.012) 

0.035 (0.011) 

0.040 (0.011) 

56 


International Avg. 

0.019 (0.011) 

0.024 (0.010) 

0.030 (0.010) 

27 

International Std. Dev. 

0.013 (0.003) 

0.017 (0.003) 

0.022 (0.003) 

24 


( ) Standard errors appear in parentheses. 
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RELATIONSHIPS AMONG DIRECT AND TOTAL INDIRECT EFFECTS The results presented 
above suggest that the strength of the direct and the total indirect effects of 
Parental Education are more or less independent on a particular subject matter 
domain while the effects seem similar across the three domains. In order to 
obtain more precise information about the pattern of relations among direct and 
total indirect effects across domains, correlations among the effect estimates for 
the 37 participants have been computed (see Exhibit 4.11). 


Exhibit 4.11: Correlations among Direct and Total Indirect Effects of Parental 
Education on Mathematics, Science, and Reading 


Direct Effects S Total Indirect Effects 


Direct Effects 

Mathematics 


Science 


Reading 


Mathematics 


Science 


Reading 

Mathematics 

1.00 











Science 

0.88 


1.00 









Reading 

0.88 


0.98 


1.00 







Indirect Effects 

Mathematics 

0.18 


0.06 


0.08 


1.00 





Science 

0.15 


0.06 


0.08 


0.94 


1.00 



Reading 

0.23 


0.13 


0.16 


0.90 


0.95 


1.00 


The correlations among the direct effects were very large, and particularly 
so for science and reading (r = 0.98). The correlations between the direct effect 
of Parental Education on mathematics and the effects on science and reading 
were somewhat smaller (0.88). The correlations among the total indirect effects 
showed a similar pattern, with the correlation for reading and science being 
the largest (r = 0.95) and the correlation for mathematics and reading being 
the smallest (r = 0.90). The correlation among the total indirect effects for 
mathematics and science also was large (r = 0.94). 
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The correlations among the direct and indirect effects of Parental 
Education were all close to zero, and none of these was significant. The absence 
of correlations between direct and total indirect effects implies that countries 
can have all possible combinations of large and small estimates of direct and 
indirect effects. These results indicate that there are different mechanisms at 
work behind the direct effects and the indirect effects. The direct effects are 
effects for which no testable explanatory mechanism has yet been proposed. 
It is, however, interesting to note the very high correlation between the direct 
effects for science and reading, which suggests that reading skills are important 
for achievement in the science domain. 

Direct Effects of Parental Education and Gender on the Mediating Variables 
This section will primarily focus on the indirect effects, because a closer analysis 
of these effects can inform us about the mechanisms through which Parental 
Education and Gender influence the achievement outcomes in the three 
domains. 

The indirect effects are created by two or more direct effects, linking the 
independent variables Parental Education and Gender and the dependent 
variables (i.e., the fourth grade student achievement measures). Because the 
indirect effect is a function of the product of the path coefficients involved in 
the path, a description of which direct relations are small and which are large is 
important for understanding the indirect effects. In the description of indirect 
effects we are, in particular, interested in the relations among variables which 
build what we have labeled the Main Path, i.e., the path from Parental Education 
to achievement via Books, Activity, and Ability. 

Below, we analyze the direct effects of Parental Education and Gender on 
the mediating variables of the path model (i.e., Books, Activity, NumLitAct, 
Ability and NumLitAb), and then we analyze the pattern of interrelations among 
these mediating variables. Estimated direct effects of Parental Education on the 
mediating variables are presented first, followed by the results for Gender. 
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DIRECT EFFECTS OF PARENTAL EDUCATION Exhibit 4.12 presents the estimated 
standardized direct effects of Parental Education on the mediating variables 
in the path model. Discussion begins by focusing on the relation between 
Parental Education and Books, which is the first link in the Main Path. 

As indicated in this Exhibit, the mean standardized regression coefficient 
((3) was 0.48 (sd = 0 . 09 ). This is a substantial relationship, though there was 
variation across countries. The highest relationships were observed in Elungary, 
Romania, Portugal, Iran, Spain, and the Slovak Republic, and the lowest 
relationships were observed in Qatar, the Emirate of Dubai, the Canadian 
province of Quebec, and Oman. There does seem to be an over-representation 
of East European countries in the group with high relationships, though there 
are exceptions; for example, the Russian Federation and the Czech Republic 
both had values below the mean (|3 = 0.41 and (3 = 0 . 45 , respectively). 

The mean (3 coefficient for the relationship between Parental Education and 
Activity was 0.05 (sd = 0 . 08 ) (see Exhibit 4 . 12 ). This is a small estimate, which 
suggests that the effects of Parental Education only to a small extent are due to 
direct effects of Parental Education on Activity. There were, however, differences 
across countries. While non-significant relationships were observed for many 
countries, two had significant negative relationships (Botswana and Morocco) 
and 16 had significant positive relationships, the highest of which were observed 
for Malta, Oman, Finland, Italy, Elong Kong SAR, and Sweden. 

It will be remembered that the NumLitAct variable is bipolar, such that 
positive values represent a relatively stronger emphasis on literacy activities 
than on numeracy while negative values represent a stronger emphasis on 
numeracy activities than on literacy activities. The mean direct effect of Parental 
Education on NumLitAct across the 37 participants was 0.01 (sd = 0 . 07 ). Thus, 
as was observed also in the Common model, there is no general effect that holds 
across countries. There were, however, substantial country differences. Four 
countries had significant negative relations: Iran, Honduras, Oman, and Chinese 
Taipei. In other words, in these countries, highly educated parents tended to 
put more emphasis on numeracy than on literacy activities. Seven countries had 
significant positive relations: Sweden, Finland, Slovenia, Norway, Morocco, the 
Czech Republic, and Saudi Arabia. In these countries, that is, highly educated 
parents placed more emphasis on literacy-oriented activities than on activities 
that were numeracy-oriented. 

The mean of the (3 coefficients for the regression of Ability on Parental 
Education was .02 (sd = 0 . 05 ). For one country (Hong Kong SAR) there was 
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Exhibit 4.1 2: Standardized Direct Effects of Parental Education on the Mediating Variables 


TIMSS frPIRLS 
2011 


Country 

Books 

Activity 

NumLitAct 

Ability 

NumLitAb [' 

Australia 

0.401 (0.027) 

0.025 (0.031) 

0.025 (0.055) 

0.045 (0.026) 

-0.044 (0.028) 

Austria 

0.508 (0.016) 

-0.004 (0.021) 

0.039 (0.050) 

0.041 (0.025) 

-0.008 (0.025) 

Azerbaijan 

0.385 (0.025) 

0.113 (0.029) 

-0.047 (0.044) 

0.063 (0.024) 

-0.088 (0.027) 

Chinese Taipei 

0.549 (0.016) 

0.091 (0.022) 

-0.093 (0.033) 

0.015 (0.020) 

-0.024 (0.029) 

Croatia 

0.525 (0.020) 

-0.016 (0.024) 

0.010 (0.040) 

-0.012 (0.021) 

-0.024 (0.024) 

Czech Republic 

0.452 (0.021) 

-0.096 (0.026) 

0.095 (0.044) 

-0.004 (0.023) 

-0.018 (0.024) 

Finland 

0.406 (0.022) 

-0.039 (0.023) 

0.122 (0.050) 

0.099 (0.024) 

-0.005 (0.024) 

Georgia 

0.521 (0.022) 

-0.008 (0.025) 

-0.019 (0.043) 

-0.016 (0.023) 

-0.104 (0.029) 

Germany 

0.545 (0.019) 

-0.016 (0.026) 

-0.003 (0.053) 

-0.009 (0.027) 

-0.064 (0.028) 

Hong Kong SAR 

0.540 (0.023) 

0.041 (0.025) 

0.017 (0.030) 

-0.073 (0.025) 

0.059 (0.030) 

Hungary 

0.688 (0.015) 

-0.067 (0.025) 

-0.016 (0.044) 

0.028 (0.025) 

-0.086 (0.028) 

Iran, Islamic Rep. of 

0.621 (0.022) 

0.032 (0.025) 

-0.22 (0.054) 

-0.038 (0.026) 

-0.104 (0.029) 

Ireland 

0.473 (0.022) 

-0.015 (0.025) 

0.010 (0.042) 

0.024 (0.025) 

-0.083 (0.028) 

Italy 

0.487 (0.016) 

0.000 (0.024) 

-0.008 (0.034) 

-0.039 (0.021) 

-0.016 (0.024) 

Lithuania 

0.507 (0.021) 

-0.026 (0.023) 

0.039 (0.039) 

0.073 (0.021) 

0.047 (0.024) 

Malta 

0.511 (0.023) 

0.069 (0.026) 

-0.059 (0.035) 

-0.018 (0.025) 

0.113 (0.028) 

Morocco 

0.417 (0.030) 

0.142 (0.028) 

0.095 (0.032) 

0.092 (0.017) 

-0.026 (0.027) 

Northern Ireland 

0.506 (0.028) 

-0.007 (0.035) 

-0.057 (0.050) 

-0.03 (0.030) 

-0.09 (0.036) 

Norway 

0.471 (0.022) 

0.037 (0.029) 

0.101 (0.047) 

-0.026 (0.024) 

-0.064 (0.033) 

Oman 

0.361 (0.019) 

0.150 (0.024) 

-0.101 (0.042) 

0.032 (0.017) 

0.073 (0.023) 

Poland 

0.564 (0.017) 

-0.012 (0.029) 

0.060 (0.037) 

0.030 (0.024) 

-0.048 (0.030) 

Portugal 

0.639 (0.020) 

-0.026 (0.030) 

0.020 (0.063) 

-0.037 (0.039) 

-0.078 (0.042) 

Qatar 

0.301 (0.033) 

0.135 (0.027) 

-0.056 (0.045) 

0.049 (0.020) 

-0.044 (0.025) 

Romania 

0.649 (0.021) 

0.272 (0.036) 

0.077 (0.052) 

0.070 (0.027) 

-0.125 (0.033) 

Russian Federation 

0.405 (0.022) 

0.075 (0.027) 

0.005 (0.040) 

0.099 (0.020) 

-0.032 (0.031) 

Saudi Arabia 

0.405 (0.025) 

0.086 (0.035) 

0.093 (0.045) 

0.049 (0.027) 

-0.1 (0.031) 

Singapore 

0.423 (0.018) 

0.142 (0.016) 

-0.007 (0.025) 

0.090 (0.018) 

0.043 (0.019) 

Slovak Republic 

0.569 (0.021) 

0.036 (0.037) 

-0.067 (0.034) 

-0.002 (0.021) 

-0.015 (0.027) 

Slovenia 

0.495 (0.020) 

-0.012 (0.026) 

0.109 (0.042) 

-0.037 (0.024) 

-0.063 (0.027) 

Spain 

0.570 (0.020) 

-0.022 (0.028) 

-0.019 (0.044) 

0.040 (0.025) 

0.049 (0.027) 

Sweden 

0.486 (0.019) 

0.088 (0.027) 

0.128 (0.049) 

0.088 (0.031) 

-0.053 (0.029) 

United Arab Emirates 

0.413 (0.014) 

0.119 (0.015) 

0.041 (0.022) 

0.028 (0.013) 

0.117 (0.019) 

Sixth Grade Countries 

Botswana 

0.392 (0.033) 

0.224 (0.024) 

-0.069 (0.040) 

0.122 (0.022) 

-0.116 (0.028) 

Honduras 

0.363 (0.053) 

0.193 (0.038) 

-0.136 (0.046) 

0.007 (0.024) 

-0.124 (0.035) 

Benchmarking Participants 

Quebec, Canada 

0.358 (0.026) 

0.015 (0.023) 

0.079 (0.042) 

0.008 (0.025) 

-0.106 (0.025) 

Abu Dhabi, UAE 

0.441 (0.022) 

0.116 (0.026) 

0.061 (0.039) 

0.061 (0.022) 

0.097 (0.029) 

Dubai, UAE 

0.343 (0.026) 

0.126 (0.019) 

0.022 (0.035) 

-0.014 (0.015) 

0.147 (0.027) 


International Avg. 

0.494 (0.021) 

0.092 (0.027) 

0.060 (0.042) 

0.056 (0.024) 

0.072 (0.028) 

International Std. Dev. 

0.089 (0.004) 

0.064 (0.005) 

0.042 (0.009) 

0.028 (0.005) 

0.031 (0.005) 


( ) Standard errors appear in parentheses. 
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a significant negative relationships between Ability and Parental Education, 
while for a dozen countries there were significant positive relations. The highest 
relationships were observed for Botswana, the Russian Federation, Finland, 
Morocco, Singapore, and Sweden. 

One would expect that students with more highly educated parents also 
are better able to perform numeracy and literacy tasks at school start. There 
may, however, be many reasons for why this relationship does not appear for 
all countries. One reason may be that the effect of Parental Education only is 
indirect, via Books and Activity. Another reason may be that parents with more 
education evaluate their children’s task performance against stricter standards. 
Yet another reason may be that, in some educational systems, school begins 
at such an early age that the students have not yet developed much of the 
numeracy and literacy skills asked about. The latter hypothesis may be tested 
by investigating how the level of relationship between Parental Education and 
Ability varies as a function of the age of the students, given that the students in 
almost all cases were assessed at the fourth grade (exceptions were Botswana 
and Elonduras where students were assessed at Grade 6, and Malta, where 
students were assessed at Grade 5). The relationship between student mean age 
at country level and the (3 coefficient was .40, which supports the hypothesis 
that student age at the time of beginning primary school is of importance, with 
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respect to the level of numeracy and literacy skills that can be demonstrated at 
that time. 

The mean of the (3 coefficients for the relationship between Parental 
Education and NumLitAb (i.e., the tendency for the parents to assess the child 
as relatively stronger in literacy tasks then numeracy tasks on the one hand) 
was -0.03 (sd = 0.07). For about a dozen countries, the parents with a high 
level of education rated numeracy skills higher than literacy skills. This was 
most pronounced in Romania, Honduras, Botswana, the Canadian province of 
Quebec, Georgia, and Iran. For about half a dozen countries, the parents with 
a high level of education rated literacy skills higher than numeracy skills. This 
was most pronounced in the United Arab Emirates, Malta, and Oman. 

In summary, there was a very strong direct effect of Parental Education 
on Books. For no other variable in the path model was there a noteworthy 
general effect of Parental Education. Judging from the results obtained in the 
Common model, the main reason for this is that the effect of Parental Education 
on the variables further down the chain is mediated via Books. However, it also 
may be noted that, for the majority of the relationships investigated, there was 
heterogeneity in the pattern of results for different countries, which will be 
discussed in the analyses of models for different countries. 
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DIRECT EFFECTS OF GENDER Exhibit 4.13 presents estimates of direct effects of 
Gender on the mediating variables in the path model. 

In comparison with Parental Education, there were fewer direct effects of 
Gender, indicating that parents tend to interact in similar ways with boys and 
girls. There were, however, several interesting exceptions to this pattern. 

First, there was a weak significant positive effect of Gender on Books in 
about a dozen countries (e.g., Lithuania, Ireland, the Emirate of Dubai, Malta, 
Sweden, Singapore, Slovenia, and Iran). Thus, in these countries, the parents 
reported a somewhat higher frequency of books when the child is a girl than 
when the child is a boy. The significant effects were between .05 and .07. 

For the Activity variable, a significant Gender effect also was found in 
some cases. For example, in Malta, a higher level of activity was reported when 
the child was a boy; however, for about ten countries, a higher level of activity 
was reported for girls (e.g., Saudi Arabia, the Emirate of Abu Dhabi, Morocco, 
Oman, and Austria). 

In most countries, there was a considerable effect of Gender for the 
NumLitAct variable, in such a way that more emphasis on literacy activities 
than on numeracy activities was reported for girls than for boys. The mean 
effect was 0.16 (sd = 07). For two countries only (Morocco and Saudi Arabia) 
was there no significant effect. The countries with the strongest direct effect of 
gender ((3 > .24) were Norway, Lithuania, Sweden, the Canadian province of 
Quebec, Germany, Finland, and Poland. 

For the Ability variable, there was only a small mean effect of 0.03 
(sd = 0.04), but there were a small number of countries where boys were rated 
higher in ability (Austria and Azerbaijan), and about a dozen countries were 
girls were rated as having better skills in doing literacy and numeracy tasks. 
Countries with the largest gender effect were Northern Ireland, Saudi Arabia, 
Hong Kong SAR, Chinese Taipei, Finland, Norway, and Singapore. 

For the NumLitAb variable, there was a direct effect of Gender, the mean 
effect being 0.07 (sd = 0.05). The positive effect implies that girls were assessed 
as being relatively better at doing literacy tasks than at doing numeracy tasks. 
There was a significant positive effect in most countries, with the largest effects 
being observed for Sweden, Croatia, the Canadian province of Quebec, Slovenia, 
and Norway. 

In summary, the results show fewer and smaller direct effects of Gender 
than of Parental Education. However, in almost all countries, the parents 
reported a stronger emphasis on literacy activities than on numeracy activities 


TIMSS AND PIRLS 201 1 RELATIONSHIPS REPORT 

228 CHAPTER 4 



TIMSS & PIRLS 

International Study Center 

Lynch School of Education, Boston College 



TIMSS fr PIRLS 
2011 


Exhibit 4.1 3: Standardized Direct Effects of Gender on the Mediating Variables 


Country 

Books 

Activity 

NumLitAct 

Ability 

NumLitAb 1 

Australia 

-0.016 (0.025) 

0.031 (0.026) 

0.153 (0.042) 

0.035 (0.024) 

0.134 (0.027) 

Austria 

0.036 (0.017) 

0.046 (0.016) 

0.227 (0.035) 

-0.054 (0.023) 

0.129 (0.026) 

Azerbaijan 

-0.012 (0.019) 

-0.001 (0.019) 

0.215 (0.036) 

-0.048 (0.022) 

0.032 (0.026) 

Chinese Taipei 

0.015 (0.015) 

0.022 (0.015) 

0.177 (0.025) 

0.079 (0.020) 

0.070 (0.022) 

Croatia 

0.036 (0.015) 

0.012 (0.017) 

0.195 (0.033) 

0.051 (0.019) 

0.158 (0.019) 

Czech Republic 

0.009 (0.021) 

0.043 (0.021) 

0.074 (0.035) 

0.024 (0.020) 

0.042 (0.022) 

Finland 

0.017 (0.019) 

0.032 (0.021) 

0.243 (0.033) 

0.077 (0.021) 

0.114 (0.021) 

Georgia 

-0.002 (0.018) 

0.013 (0.018) 

0.084 (0.035) 

0.064 (0.016) 

0.095 (0.022) 

Germany 

0.025 (0.018) 

0.028 (0.019) 

0.257 (0.041) 

-0.005 (0.023) 

0.089 (0.029) 

Hong Kong SAR 

0.007 (0.022) 

-0.004 (0.019) 

0.147 (0.021) 

0.083 (0.016) 

0.008 (0.020) 

Hungary 

0.003 (0.014) 

0.027 (0.016) 

0.138 (0.024) 

-0.003 (0.017) 

0.055 (0.020) 

Iran, Islamic Rep. of 

0.046 (0.023) 

-0.016 (0.024) 

0.161 (0.038) 

0.038 (0.020) 

0.037 (0.022) 

Ireland 

0.066 (0.023) 

-0.029 (0.021) 

0.146 (0.032) 

0.031 (0.020) 

0.130 (0.031) 

Italy 

0.025 (0.017) 

0.032 (0.018) 

0.196 (0.030) 

-0.013 (0.019) 

0.097 (0.023) 

Lithuania 

0.075 (0.017) 

0.028 (0.022) 

0.307 (0.031) 

0.041 (0.023) 

0.130 (0.024) 

Malta 

0.061 (0.019) 

-0.046 (0.020) 

0.138 (0.031) 

0.047 (0.021) 

0.055 (0.024) 

Morocco 

0.019 (0.018) 

0.060 (0.026) 

-0.011 (0.030) 

0.036 (0.017) 

0.014 (0.025) 

Northern Ireland 

0.055 (0.029) 

0.042 (0.029) 

0.212 (0.037) 

0.098 (0.026) 

0.053 (0.033) 

Norway 

0.020 (0.021) 

0.024 (0.028) 

0.323 (0.038) 

0.068 (0.025) 

0.138 (0.029) 

Oman 

0.003 (0.015) 

0.055 (0.015) 

0.133 (0.025) 

0.023 (0.015) 

0.068 (0.016) 

Poland 

0.030 (0.017) 

0.042 (0.017) 

0.236 (0.029) 

0.005 (0.018) 

0.049 (0.023) 

Portugal 

0.018 (0.021) 

0.037 (0.021) 

0.160 (0.038) 

-0.011 (0.021) 

0.058 (0.022) 

Qatar 

0.040 (0.023) 

0.030 (0.021) 

0.084 (0.038) 

0.026 (0.019) 

0.029 (0.027) 

Romania 

0.031 (0.014) 

0.023 (0.016) 

0.076 (0.027) 

0.047 (0.016) 

0.004 (0.026) 

Russian Federation 

0.023 (0.017) 

0.044 (0.018) 

0.120 (0.029) 

0.035 (0.017) 

0.051 (0.022) 

Saudi Arabia 

0.070 (0.041) 

0.134 (0.027) 

0.033 (0.035) 

0.093 (0.030) 

0.070 (0.026) 

Singapore 

0.050 (0.018) 

0.016 (0.013) 

0.117 (0.019) 

0.067 (0.013) 

0.040 (0.015) 

Slovak Republic 

0.040 (0.016) 

0.039 (0.018) 

0.153 (0.031) 

-0.022 (0.015) 

0.059 (0.018) 

Slovenia 

0.049 (0.017) 

0.009 (0.017) 

0.181 (0.034) 

0.046 (0.023) 

0.144 (0.025) 

Spain 

-0.007 (0.018) 

0.016 (0.017) 

0.204 (0.033) 

-0.001 (0.020) 

0.107 (0.021) 

Sweden 

0.050 (0.021) 

0.035 (0.021) 

0.281 (0.039) 

0.063 (0.025) 

0.177 (0.030) 

United Arab Emirates 

0.008 (0.015) 

0.032 (0.012) 

0.107 (0.020) 

0.023 (0.012) 

0.028 (0.020) 

Sixth Grade Countries 

Botswana 

-0.002 (0.020) 

0.039 (0.019) 

0.152 (0.029) 

0.028 (0.018) 

0.006 (0.022) 

Honduras 

0.000 (0.023) 

-0.02 (0.022) 

0.113 (0.038) 

0.000 (0.027) 

-0.009 (0.031) 

Benchmarking Participants 

Quebec, Canada 

0.017 (0.023) 

0.012 (0.019) 

0.267 (0.030) 

0.011 (0.020) 

0.150 (0.026) 

Abu Dhabi, UAE 

0.018 (0.023) 

0.076 (0.021) 

0.127 (0.035) 

0.031 (0.021) 

0.048 (0.029) 

Dubai, UAE 

0.065 (0.026) 

0.024 (0.013) 

0.150 (0.030) 

0.034 (0.021) 

-0.005 (0.031) 


International Avg. 

0.033 (0.019) 

0.035 (0.020) 

0.170 (0.032) 

0.050 (0.020) 

0.077 (0.024) 

International Std. Dev. 

0.021 (0.005) 

0.023 (0.004) 

0.070 (0.006) 

0.024 (0.004) 

0.047 (0.004) 


( ) Standard errors appear in parentheses. 
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SOURCE: lEA's Trends in International Mathematics and Science Study and Progress in International Reading Literacy Study - TIMSS and PIRLS 201 1 










when their child was a girl than when their child was a boy. Furthermore, in 
most countries, the parents assessed girls’ literacy skills to be stronger than their 
numeracy skills. There also was a tendency for the parents to report more books 
in the home for girls, and also a tendency towards a higher level of activity with 
girls. 

Direct Effects Among the Mediating Variables 

This section looks closer at the relationships among the mediating variables: 
Books, and the Activity and Ability variables. These relationships are, of course, 
the same when we investigate effects of Parental Education and Gender. 

EFFECTS OF BOOKS Exhibit 4.14 presents standardized direct effects of Books on 
the variables in the path model. 

As is presented in Exhibit 4. 14, there was a large direct effect of Books on 
Activity, the mean (3 being 0.34 (sd = 0.07). There also was a sizeable variation 
in the strength of the relationship across countries. The smallest effects were 
around .20 and were observed for Elonduras, Botswana, Morocco, the Czech 
Republic, Italy, and Azerbaijan. The largest effects were higher than 0.40 and 
were observed for Portugal, Chinese Taipei, Malta, Austria, Georgia, and Ireland. 

For the direct effect of Books on the variable NumLitAct, the mean was 
0.09 (sd = 0.09), indicating that parents who reported a larger number of books 
in the home also tended to report that activities were more literacy oriented than 
numeracy oriented. For two countries (Georgia and Elungary) the effect was 
negative and significant, while for around 20 countries the effect was positive 
and significant. The largest positive effects were observed for Italy, Chinese 
Taipei, Lithuania, Germany, and Sweden. 

The mean direct effect of Books on Ability was 0.0, but there was large 
variability across countries (sd = 0.10). For eight countries the effect was 
negative and significant, most markedly so in Austria, Ireland, Germany, and 
Australia. For ten countries the effect was positive and significant, with the 
strongest relationships in Singapore, Russian Federation and Lithuania. 

For the direct effect of Books on the variable NumLitAb, the mean estimate 
of (3 was close to 0, and for only one country (Portugal) was there a weak 
significant positive effect. 

The Books variable had strong direct effects on the three achievement 
variables: 0.22 for mathematics, 0.24 for science and 0.23 for reading. However, 
there also was considerable variation across countries (sd between .09 and .10). 
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Exhibit 4.14: Standardized Direct Effect of Books on the Variables in the Path Model 


Country 

Activity 

NumLitAct 

Ability 

NumLitAb 

Mathematics 

Science 

Reading 1] 

Australia 

0.381 (0.028) 

0.118 (0.060) 

-0.147 (0.030) 

0.036 (0.027) 

0.226 (0.033) 

0.270 (0.034) 

0.233 (0.034) 

Austria 

0.417 (0.020) 

0.082 (0.061) 

-0.245 (0.029) 

0.034 (0.026) 

0.441 (0.035) 

0.447 (0.031) 

0.422 (0.029) 

Azerbaijan 

0.262 (0.034) 

0.073 (0.046) 

0.069 (0.026) 

0.031 (0.026) 

0.083 (0.038) 

0.095 (0.039) 

0.061 (0.036) 

Chinese Taipei 

0.440 (0.020) 

0.213 (0.038) 

0.069 (0.029) 

0.036 (0.022) 

0.227 (0.023) 

0.237 (0.028) 

0.179 (0.023) 

Croatia 

0.329 (0.024) 

0.081 (0.039) 

0.055 (0.022) 

0.026 (0.019) 

0.221 (0.023) 

0.255 (0.027) 

0.246 (0.024) 

Czech Republic 

0.257 (0.026) 

0.109 (0.051) 

-0.001 (0.028) 

0.034 (0.022) 

0.280 (0.024) 

0.317 (0.026) 

0.297 (0.026) 

Finland 

0.328 (0.021) 

0.149 (0.060) 

0.078 (0.035) 

0.037 (0.021) 

0.194 (0.035) 

0.264 (0.032) 

0.237 (0.031) 

Georgia 

0.415 (0.028) 

-0.177 (0.045) 

0.086 (0.027) 

0.033 (0.022) 

0.208 (0.038) 

0.222 (0.042) 

0.220 (0.036) 

Germany 

0.312 (0.027) 

0.192 (0.059) 

-0.16 (0.030) 

0.038 (0.029) 

0.375 (0.034) 

0.412 (0.037) 

0.373 (0.033) 

Hong Kong SAR 

0.392 (0.025) 

0.133 (0.037) 

0.080 (0.028) 

0.031 (0.020) 

0.154 (0.033) 

0.165 (0.031) 

0.119 (0.032) 

Hungary 

0.381 (0.027) 

-0.157 (0.054) 

-0.04 (0.030) 

0.031 (0.020) 

0.324 (0.030) 

0.366 (0.027) 

0.307 (0.029) 

Iran, Islamic Rep. of 

0.371 (0.022) 

0.027 (0.068) 

0.004 (0.026) 

0.036 (0.022) 

0.223 (0.034) 

0.215 (0.029) 

0.201 (0.030) 

Ireland 

0.396 (0.024) 

0.157 (0.056) 

-0.169 (0.027) 

0.042 (0.031) 

0.348 (0.029) 

0.368 (0.039) 

0.378 (0.028) 

Italy 

0.258 (0.028) 

0.221 (0.038) 

-0.09 (0.027) 

0.030 (0.023) 

0.203 (0.028) 

0.267 (0.033) 

0.261 (0.026) 

Lithuania 

0.277 (0.027) 

0.195 (0.041) 

0.126 (0.029) 

0.033 (0.024) 

0.207 (0.025) 

0.220 (0.028) 

0.229 (0.026) 

Malta 

0.418 (0.029) 

0.087 (0.045) 

-0.012 (0.031) 

0.034 (0.024) 

0.201 (0.033) 

0.226 (0.029) 

0.193 (0.030) 

Morocco 

0.243 (0.031) 

0.093 (0.040) 

0.062 (0.022) 

0.034 (0.025) 

0.032 (0.028) 

0.056 (0.028) 

0.034 (0.026) 

Northern Ireland 

0.331 (0.036) 

0.168 (0.059) 

-0.131 (0.035) 

0.049 (0.033) 

0.268 (0.042) 

0.332 (0.047) 

0.268 (0.040) 

Norway 

0.350 (0.032) 

0.085 (0.055) 

-0.035 (0.029) 

0.042 (0.029) 

0.227 (0.038) 

0.304 (0.033) 

0.291 (0.034) 

Oman 

0.310 (0.022) 

0.079 (0.036) 

0.028 (0.020) 

0.022 (0.016) 

0.130 (0.021) 

0.096 (0.021) 

0.126 (0.022) 

Poland 

0.357 (0.033) 

0.164 (0.045) 

-0.028 (0.027) 

0.031 (0.023) 

0.212 (0.028) 

0.211 (0.027) 

0.179 (0.029) 

Portugal 

0.489 (0.025) 

0.085 (0.072) 

0.027 (0.037) 

0.045 (0.022) 

0.269 (0.044) 

0.236 (0.049) 

0.259 (0.045) 

Qatar 

0.278 (0.029) 

0.138 (0.035) 

-0.054 (0.029) 

0.027 (0.027) 

0.228 (0.046) 

0.173 (0.043) 

0.193 (0.042) 

Romania 

0.385 (0.033) 

-0.077 (0.057) 

0.062 (0.034) 

0.038 (0.026) 

0.228 (0.036) 

0.220 (0.036) 

0.229 (0.035) 

Russian Federation 

0.378 (0.022) 

0.026 (0.041) 

0.154 (0.023) 

0.027 (0.022) 

0.144 (0.029) 

0.177 (0.029) 

0.186 (0.025) 

Saudi Arabia 

0.354 (0.041) 

0.040 (0.043) 

0.090 (0.027) 

0.032 (0.026) 

0.138 (0.053) 

0.155 (0.047) 

0.103 (0.042) 

Singapore 

0.336 (0.018) 

0.147 (0.029) 

0.195 (0.019) 

0.025 (0.015) 

0.208 (0.020) 

0.251 (0.020) 

0.235 (0.018) 

Slovak Republic 

0.318 (0.031) 

0.101 (0.045) 

0.023 (0.022) 

0.030 (0.018) 

0.358 (0.032) 

0.366 (0.032) 

0.348 (0.027) 

Slovenia 

0.324 (0.025) 

0.060 (0.045) 

0.038 (0.027) 

0.029 (0.025) 

0.295 (0.026) 

0.327 (0.023) 

0.309 (0.022) 

Spain 

0.378 (0.028) 

0.140 (0.051) 

0.075 (0.031) 

0.031 (0.021) 

0.214 (0.033) 

0.249 (0.035) 

0.191 (0.033) 

Sweden 

0.267 (0.028) 

0.189 (0.049) 

-0.051 (0.034) 

0.039 (0.030) 

0.315 (0.035) 

0.404 (0.032) 

0.310 (0.031) 

United Arab Emirates 

0.325 (0.014) 

0.114 (0.025) 

-0.037 (0.019) 

0.018 (0.020) 

0.241 (0.024) 

0.203 (0.024) 

0.252 (0.023) 

Sixth Grade Countries 

Botswana 

0.212 (0.031) 

-0.036 (0.038) 

0.061 (0.022) 

0.023 (0.022) 

0.054 (0.024) 

0.047 (0.023) 

0.055 (0.023) 

Honduras 

0.162 (0.042) 

-0.011 (0.037) 

-0.02 (0.025) 

0.028 (0.031) 

0.041 (0.030) 

0.064 (0.029) 

0.062 (0.031) 

Benchmarking Participants 

Quebec, Canada 

0.368 (0.027) 

0.149 (0.051) 

-0.067 (0.030) 

0.033 (0.026) 

0.206 (0.031) 

0.257 (0.034) 

0.183 (0.030) 

Abu Dhabi, UAE 

0.302 (0.026) 

0.102 (0.044) 

-0.02 (0.036) 

0.032 (0.029) 

0.183 (0.045) 

0.129 (0.047) 

0.193 (0.048) 

Dubai, UAE 

0.354 (0.021) 

0.108 (0.043) 

-0.091 (0.026) 

0.026 (0.031) 

0.356 (0.030) 

0.335 (0.029) 

0.353 (0.026) 


International Avg. 

0.346 (0.027) 

0.120 (0.048) 

0.073 (0.028) 

0.033 (0.024) 

0.232 (0.032) 

0.253 (0.032) 

0.233 (0.030) 

International Std. Dev. 

0.060 (0.006) 

0.054 (0.011) 

0.047 (0.005) 

0.006 (0.004) 

0.084 (0.008) 

0.094 (0.007) 

0.088 (0.006) 


( ) Standard errors appear in parentheses. 
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The correlations between the (3s for the three relations were from .94 to .96. For 
the following countries a strong direct effect of Books on achievement in all 
three domains was observed: Austria, Germany, the Slovak Republic, Ireland, 
Sweden, the Emirate of Dubai, and Hungary. 

In summary, the results show, as expected, a substantial positive direct 
effect of Books on Activity, but also a large variation across countries. For the 
majority of countries there also was a positive effect of Books on NumLitAct, 
which implies that homes with many books tended to be engaged in more 
literacy than numeracy activities. There was no overall effect of Books on Ability, 
though there were differences across country. 

EFFECTS OF THE ACTIVITY VARIABLES Exhibit 4.15 presents the standardized direct 
effects of the activity variables on the other variables in the path model. 

As noted in Exhibit 4.15, the Activity variable had a direct effect of 0.40 
(sd = 0.08) on Ability, and the positive effect was significant in all countries. 
The smallest effects (.31 or lower) were observed for Finland, Spain, Hong Kong 
SAR, and Croatia, while the largest effects (.51 or higher) were observed for 
Romania, Morocco, Azerbaijan, and the Slovak Republic. 

The average effect of the Activity variable on NumLitAb was close to 0, but 
there was variation across countries (sd = 0.07). For five countries there was 
a significant negative effect, implying that a high level of Activity was related 
to higher numeracy skills than literacy skills at school start. Interestingly, 
these five countries were all are East European countries: Romania, the 
Russian Federation, Hungary, the Slovak Republic, and the Czech Republic. 
Three countries, Morocco, Singapore, and Ireland, had a significant positive 
relationship. 

The average direct effect of Activity was close to zero for all three 
achievement measures, but there was variability across countries (sd between 
0.06 and 0.07). The correlations between the (3s for the three relations were 
.92 to .93. For mathematics there were significant negative direct effects for 
18 countries, with the largest effects being observed for Morocco, Singapore, 
Slovenia, the Russian Federation, and Norway. Significant positive direct effects 
were observed for the United Arab Emirates and the two Emirates of Abu 
Dhabi and Dubai. For science there were significant negative relationships for 
eight countries, including all of the aforementioned, with significant negative 
relationships between Activity and mathematics achievement. There were 
significant positive effects on science achievement in six countries, among 
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which were those with significant positive effects on mathematics achievement. 
There was a negative direct effect of Activity on reading for five countries, and 
a positive effect for six countries, the results largely being in agreement with 
those observed for science. 

With respect to the estimated direct effects of NumLitAct, this variable 
had a strong direct effect on the Ability variable with an average of 0.20 
(sd = 0.08). This finding implies that activities which emphasize literacy rather 
than numeracy are positively related to a high level of skills in performing both 
literacy and numeracy tasks at the beginning of primary school. While there was 
variability in the estimated (3 coefficients, they were significant and positive in 
all countries. The estimates ranged from a low of .06 in Singapore to a high of 
.40 in Azerbaijan. Other countries with large estimated direct effects included 
Austria, Hungary, Germany, Oman, and Slovenia. 

The NumLitAct variable also had a rather large direct effect on the 
NumLitAb variable, the average effect being 0.13 (sd = 0.10). Only in Georgia 
was the effect significantly negative; rather, it was positive and significant 
in most countries. These positive relations thus imply that activities which 
emphasize literacy more than numeracy are related to higher literacy skills than 
numeracy skills. 

The NumLitAct variable had an average direct effect on mathematics 
achievement that was negative (-0.07, sd = 0.06). The negative effect was 
significant for 16 countries, while the effect was non-significant for all other 
participants. The largest negative effects were observed for Finland, Morocco, 
Hungary, Iran, and Norway. At a general level, it is perhaps reasonable to expect 
that activities which emphasize literacy rather than numeracy would have a 
negative direct effect on mathematic achievement. 

The NumLitAct variable had a negative direct effect on science achievement 
in six countries: Iran, Morocco, Hungary, Romania, Croatia, Ireland, and 
Chinese Taipei. However, NumLitAct had a positive direct effect on reading 
achievement for three participants (the Canadian province of Quebec, Sweden, 
and Spain), and a negative direct effect in three participants (Iran, Romania, 
and Hungary). 

In summary, Activity had a substantial relationship with Ability, 
supporting the idea of the Main Path. There was no general effect of Activity 
on NumLitAb or on any of the three achievement variables, but there were 
small direct effects that varied across countries. As expected, NumLitAct had a 
significant effect on NumLitAb in almost all countries, even though the effect 
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Exhibit 4.15: 


Standardized Direct Effects of the Activity Variables 
on the Other Variables in the Path Model 
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2011 


^th 

Grade 


Country 

Activity 

Ability 

NumLitAb 

Mathematics 

Science 

Reading 

Australia 

0.396 (0.028) 

0.026 (0.038) 

-0.039 (0.033) 

0.015 (0.034) 

0.002 (0.034) 

Austria 

0.387 (0.022) 

0.017 (0.034) 

-0.082 (0.031) 

-0.012 (0.026) 

-0.002 (0.029) 

Azerbaijan 

0.530 (0.025) 

-0.052 (0.047) 

-0.021 (0.037) 

0.015 (0.035) 

0.009 (0.035) 

Chinese Taipei 

0.386 (0.018) 

0.039 (0.029) 

-0.022 (0.020) 

-0.038 (0.020) 

-0.033 (0.020) 

Croatia 

0.309 (0.021) 

-0.037 (0.024) 

-0.042 (0.020) 

0.000 (0.025) 

-0.028 (0.019) 

Czech Republic 

0.377 (0.019) 

-0.106 (0.029) 

-0.086 (0.023) 

-0.081 (0.025) 

-0.058 (0.023) 

Finland 

0.246 (0.026) 

-0.034 (0.026) 

-0.058 (0.023) 

-0.048 (0.029) 

-0.056 (0.023) 

Georgia 

0.490 (0.023) 

-0.042 (0.038) 

-0.093 (0.033) 

-0.044 (0.034) 

-0.033 (0.033) 

Germany 

0.388 (0.023) 

0.056 (0.030) 

-0.087 (0.026) 

-0.038 (0.025) 

-0.03 (0.027) 

Hong Kong SAR 

0.303 (0.022) 

0.012 (0.022) 

-0.043 (0.022) 

-0.049 (0.022) 

-0.035 (0.032) 

Hungary 

0.465 (0.025) 

-0.12 (0.044) 

-0.006 (0.024) 

-0.02 (0.031) 

0.004 (0.030) 

Iran, Islamic Rep. of 

0.504 (0.030) 

-0.027 (0.031) 

-0.063 (0.026) 

-0.02 (0.028) 

-0.025 (0.024) 

Ireland 

0.374 (0.019) 

0.067 (0.024) 

-0.007 (0.027) 

0.032 (0.032) 

0.008 (0.028) 

Italy 

0.382 (0.020) 

0.029 (0.030) 

-0.016 (0.029) 

0.036 (0.025) 

0.016 (0.024) 

Lithuania 

0.369 (0.019) 

-0.033 (0.028) 

-0.085 (0.024) 

-0.101 (0.023) 

-0.098 (0.022) 

Malta 

0.334 (0.022) 

0.021 (0.027) 

0.051 (0.027) 

0.051 (0.025) 

0.092 (0.026) 

Morocco 

0.569 (0.034) 

0.162 (0.039) 

-0.274 (0.045) 

-0.204 (0.041) 

-0.174 (0.037) 

Northern Ireland 

0.397 (0.025) 

0.033 (0.044) 

-0.041 (0.034) 

-0.044 (0.037) 

0.008 (0.036) 

Norway 

0.466 (0.023) 

0.009 (0.044) 

-0.117 (0.033) 

-0.06 (0.035) 

-0.063 (0.037) 

Oman 

0.435 (0.019) 

0.047 (0.025) 

0.038 (0.022) 

0.047 (0.022) 

0.042 (0.020) 

Poland 

0.470 (0.022) 

0.014 (0.034) 

-0.05 (0.023) 

-0.029 (0.025) 

0.005 (0.025) 

Portugal 

0.356 (0.023) 

0.007 (0.034) 

-0.071 (0.031) 

-0.009 (0.036) 

-0.073 (0.030) 

Qatar 

0.352 (0.020) 

-0.002 (0.024) 

-0.005 (0.029) 

0.037 (0.025) 

0.009 (0.023) 

Romania 

0.587 (0.032) 

-0.213 (0.045) 

-0.088 (0.042) 

-0.032 (0.039) 

-0.026 (0.031) 

Russian Federation 

0.467 (0.023) 

-0.121 (0.036) 

-0.125 (0.029) 

-0.105 (0.036) 

-0.089 (0.027) 

Saudi Arabia 

0.425 (0.030) 

0.013 (0.036) 

0.010 (0.037) 

0.004 (0.039) 

0.030 (0.030) 

Singapore 

0.363 (0.016) 

0.069 (0.021) 

-0.134 (0.017) 

-0.104 (0.014) 

-0.088 (0.015) 

Slovak Republic 

0.511 (0.028) 

-0.117 (0.035) 

-0.104 (0.036) 

-0.088 (0.034) 

-0.051 (0.031) 

Slovenia 

0.383 (0.022) 

-0.047 (0.029) 

-0.131 (0.023) 

-0.079 (0.026) 

-0.062 (0.023) 

Spain 

0.288 (0.025) 

-0.022 (0.032) 

0.008 (0.028) 

0.031 (0.028) 

0.022 (0.029) 

Sweden 

0.344 (0.023) 

0.049 (0.032) 

-0.062 (0.027) 

-0.008 (0.028) 

-0.005 (0.027) 

United Arab Emirates 

0.367 (0.013) 

0.026 (0.017) 

0.056 (0.018) 

0.089 (0.018) 

0.083 (0.015) 

Sixth Grade Countries 

Botswana 

0.472 (0.029) 

0.029 (0.032) 

0.050 (0.030) 

0.068 (0.028) 

0.066 (0.023) 

Honduras 

0.419 (0.026) 

0.069 (0.039) 

-0.041 (0.041) 

-0.048 (0.040) 

-0.03 (0.039) 

Benchmarking Participants 

Quebec, Canada 

0.334 (0.024) 

0.014 (0.030) 

-0.057 (0.027) 

-0.022 (0.035) 

0.032 (0.028) 

Abu Dhabi, UAE 

0.375 (0.021) 

0.000 (0.029) 

0.064 (0.030) 

0.105 (0.027) 

0.092 (0.024) 

Dubai, UAE 

0.338 (0.016) 

0.025 (0.026) 

0.093 (0.027) 

0.111 (0.027) 

0.115 (0.023) 


International Avg. 

0.407 (0.023) 

0.039 (0.032) 

0.033 (0.028) 

0.032 (0.029) 

0.025 (0.027) 

International Std. Dev. 

0.081 (0.005) 

0.036 (0.008) 

0.023 (0.007) 

0.025 (0.007) 

0.030 (0.006) 


( ) Standard errors appear in parentheses. 
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Exhibit 4.1 5: Standardized Direct Effects of the Activity Variables 
on the Other Variables in the Path Model (Continued) 


Country 

NumLitAct 

Ability 

NumLitAb 

Mathematics 

Science 

Reading 

Australia 

0.176 (0.039) 

0.164 (0.051) 

-0.061 (0.046) 

0.008 (0.046) 

0.032 (0.043) 

Austria 

0.393 (0.046) 

0.314 (0.064) 

-0.133 (0.057) 

-0.046 (0.049) 

0.000 (0.054) 

Azerbaijan 

0.398 (0.041) 

0.157 (0.074) 

0.065 (0.057) 

0.032 (0.063) 

0.009 (0.061) 

Chinese Taipei 

0.114 (0.024) 

0.033 (0.032) 

-0.113 (0.031) 

-0.060 (0.027) 

-0.005 (0.030) 

Croatia 

0.232 (0.037) 

0.135 (0.039) 

-0.126 (0.035) 

-0.086 (0.041) 

0.002 (0.040) 

Czech Republic 

0.194 (0.039) 

0.109 (0.048) 

-0.024 (0.038) 

0.026 (0.040) 

0.074 (0.040) 

Finland 

0.189 (0.040) 

0.260 (0.050) 

-0.182 (0.044) 

-0.034 (0.053) 

0.041 (0.044) 

Georgia 

0.199 (0.033) 

-0.117 (0.049) 

-0.084 (0.038) 

-0.032 (0.038) 

-0.002 (0.040) 

Germany 

0.320 (0.045) 

0.352 (0.058) 

-0.139 (0.059) 

-0.027 (0.055) 

0.007 (0.054) 

Hong Kong SAR 

0.147 (0.029) 

0.081 (0.028) 

-0.083 (0.032) 

-0.025 (0.028) 

-0.005 (0.026) 

Hungary 

0.332 (0.036) 

0.276 (0.046) 

-0.179 (0.036) 

-0.109 (0.039) 

-0.080 (0.037) 

Iran, Islamic Rep. of 

0.203 (0.033) 

0.174 (0.044) 

-0.176 (0.039) 

-0.178 (0.037) 

-0.166 (0.035) 

Ireland 

0.173 (0.032) 

0.022 (0.043) 

-0.061 (0.035) 

-0.070 (0.034) 

-0.033 (0.033) 

Italy 

0.228 (0.035) 

0.151 (0.045) 

-0.062 (0.039) 

0.007 (0.043) 

0.063 (0.038) 

Lithuania 

0.159 (0.042) 

0.201 (0.041) 

-0.080 (0.037) 

-0.012 (0.041) 

0.014 (0.040) 

Malta 

0.155 (0.031) 

-0.042 (0.035) 

-0.018 (0.031) 

-0.034 (0.027) 

-0.022 (0.026) 

Morocco 

0.274 (0.037) 

0.243 (0.114) 

-0.180 (0.060) 

-0.137 (0.057) 

-0.118 (0.067) 

Northern Ireland 

0.153 (0.035) 

0.117 (0.052) 

-0.045 (0.047) 

-0.022 (0.052) 

0.002 (0.042) 

Norway 

0.237 (0.036) 

0.170 (0.051) 

-0.158 (0.052) 

0.022 (0.046) 

0.037 (0.047) 

Oman 

0.285 (0.029) 

0.091 (0.035) 

-0.008 (0.034) 

-0.020 (0.034) 

-0.031 (0.032) 

Poland 

0.260 (0.030) 

0.220 (0.039) 

-0.045 (0.034) 

0.054 (0.040) 

0.043 (0.034) 

Portugal 

0.139 (0.046) 

0.088 (0.043) 

-0.084 (0.037) 

-0.041 (0.035) 

-0.007 (0.038) 

Qatar 

0.158 (0.037) 

0.049 (0.040) 

-0.089 (0.029) 

-0.035 (0.032) 

-0.004 (0.032) 

Romania 

0.112 (0.030) 

0.124 (0.035) 

-0.103 (0.039) 

-0.109 (0.040) 

-0.087 (0.028) 

Russian Federation 

0.232 (0.031) 

0.045 (0.047) 

-0.013 (0.037) 

-0.013 (0.043) 

0.031 (0.036) 

Saudi Arabia 

0.201 (0.043) 

0.096 (0.055) 

0.023 (0.035) 

0.012 (0.037) 

0.004 (0.033) 

Singapore 

0.062 (0.019) 

0.045 (0.027) 

-0.083 (0.019) 

-0.039 (0.019) 

-0.017 (0.019) 

Slovak Republic 

0.208 (0.027) 

0.240 (0.035) 

-0.065 (0.036) 

0.008 (0.039) 

-0.015 (0.034) 

Slovenia 

0.281 (0.033) 

0.171 (0.052) 

-0.038 (0.037) 

0.057 (0.039) 

0.068 (0.035) 

Spain 

0.109 (0.040) 

0.076 (0.043) 

-0.015 (0.047) 

0.028 (0.041) 

0.069 (0.033) 

Sweden 

0.236 (0.051) 

0.134 (0.062) 

-0.075 (0.042) 

0.063 (0.046) 

0.133 (0.045) 

United Arab Emirates 

0.115 (0.019) 

0.095 (0.023) 

-0.001 (0.018) 

0.014 (0.021) 

0.036 (0.020) 

Sixth Grade Countries 

Botswana 

0.120 (0.042) 

0.115 (0.045) 

0.013 (0.034) 

-0.005 (0.033) 

-0.036 (0.029) 

Honduras 

0.135 (0.043) 

-0.080 (0.050) 

-0.018 (0.044) 

-0.060 (0.043) 

-0.038 (0.039) 

Benchmarking Participants 

Quebec, Canada 

0.188 (0.040) 

0.094 (0.046) 

-0.064 (0.044) 

0.068 (0.042) 

0.139 (0.043) 

Abu Dhabi, UAE 

0.139 (0.030) 

0.113 (0.036) 

-0.018 (0.030) 

-0.002 (0.031) 

0.020 (0.031) 

Dubai, UAE 

0.103 (0.028) 

0.112 (0.033) 

-0.025 (0.032) 

0.012 (0.034) 

0.039 (0.031) 


International Avg. 

0.209 (0.035) 

0.148 (0.047) 

0.044 (0.039) 

0.028 (0.040) 

0.037 (0.038) 

International Std. Dev. 

0.080 (0.007) 

0.085 (0.016) 

0.030 (0.010) 

0.020 (0.010) 

0.035 (0.011) 


( ) Standard errors appear in parentheses. 
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was relatively small. NumLitAct also had a significant direct effect on Ability 
in every country, and this relationship was not expected. This implies that, in 
homes where there is greater emphasis on literacy activities than on numeracy 
activities, both numeracy skills and literacy skills at school start are higher. It 
also was unexpected to find that NumLitAct had a significant negative effect on 
mathematics achievement in about half of the countries. There may be several 
explanations for these unexpected relationships, which will be discussed later. 

EFFECTS OF THE ABILITY VARIABLES Exhibit 4.16 presents standardized direct effects 
of the Ability variables on the three achievement variables. 

As shown in Exhibit 4.16, the average of the direct effects of Ability on 
mathematics achievement was 0.25 (sd = 0.10). For both science and reading 
the direct effects of Ability were 0.21 (sd = 0.08). The direct effect was significant 
in every country except Azerbaijan. 

The average effect of NumLitAb on mathematics achievement was close to 
zero, but there was variability across countries (sd = 0.07). Significant negative 
direct effects on mathematics achievement were observed in 14 countries. Such 
negative effects are to be expected, given that negative scores on NumLitAb 
express stronger numeracy skills than literacy skills. Significant positive effects 
were found for seven countries, with the largest effects found for the Emirate of 
Abu Dhabi, Morocco, Qatar, the United Arab Emirates, the Emirate of Dubai, 
and Oman. 

The average effect of NumLitAb on science achievement was close to 
zero, though again there was variation across countries (sd = 0.07). There 
were significant negative direct effects for twelve countries, while there were 
significant positive direct effects for eight countries. 

The average direct effect of NumLitAb on reading achievement was close 
to zero, similar to mathematics and science, but ten countries had significant 
negative effects while eleven countries had significant positive effects. The 
negative effects were rather weak. 

In summary, the expected positive direct effects of Ability on the three 
fourth grade student achievement measures were observed in all countries, 
except for one. For the NumLitAb variable, there were no general effects on 
achievement. However, both positive and negative significant effects within 
different countries were observed. 
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Exhibit 4.16: Standardized Direct Effects of the Ability Variables on Achievement, Grade 4 


TIMSS &PIRLS 
2011 


Country 

Ability 

NumLitAb 

Mathematics 

Science 

Reading 

Mathematics 

Science 

Reading 

Australia 

0.320 (0.029) 

0.262 (0.029) 

0.231 (0.027) 

-0.080 (0.027) 

-0.071 (0.027) 

-0.070 (0.026) 

Austria 

0.289 (0.039) 

0.123 (0.030) 

0.116 (0.036) 

-0.095 (0.027) 

-0.081 (0.029) 

-0.044 (0.029) 

Azerbaijan 

0.024 (0.040) 

0.028 (0.042) 

0.019 (0.047) 

-0.128 (0.028) 

-0.102 (0.027) 

-0.034 (0.027) 

Chinese Taipei 

0.301 (0.021) 

0.275 (0.022) 

0.262 (0.020) 

0.012 (0.021) 

-0.004 (0.022) 

0.019 (0.021) 

Croatia 

0.379 (0.021) 

0.243 (0.025) 

0.262 (0.020) 

-0.063 (0.024) 

-0.009 (0.023) 

0.011 (0.023) 

Czech Republic 

0.254 (0.025) 

0.172 (0.026) 

0.183 (0.024) 

-0.129 (0.023) 

-0.106 (0.026) 

-0.075 (0.025) 

Finland 

0.471 (0.021) 

0.286 (0.021) 

0.315 (0.023) 

-0.035 (0.027) 

-0.044 (0.026) 

0.025 (0.028) 

Georgia 

0.211 (0.029) 

0.198 (0.028) 

0.209 (0.025) 

-0.090 (0.027) 

-0.086 (0.030) 

-0.055 (0.023) 

Germany 

0.290 (0.035) 

0.150 (0.036) 

0.163 (0.030) 

-0.086 (0.032) 

-0.092 (0.031) 

-0.073 (0.029) 

Hong Kong SAR 

0.379 (0.023) 

0.395 (0.020) 

0.359 (0.020) 

0.033 (0.026) 

0.066 (0.027) 

0.105 (0.028) 

Hungary 

0.215 (0.028) 

0.157 (0.026) 

0.178 (0.027) 

-0.089 (0.022) 

-0.068 (0.021) 

-0.044 (0.021) 

Iran, Islamic Rep. of 

0.198 (0.024) 

0.175 (0.027) 

0.179 (0.026) 

-0.102 (0.024) 

-0.086 (0.025) 

-0.085 (0.023) 

Ireland 

0.199 (0.032) 

0.141 (0.027) 

0.163 (0.027) 

-0.041 (0.023) 

-0.055 (0.032) 

-0.025 (0.024) 

Italy 

0.205 (0.025) 

0.091 (0.031) 

0.105 (0.025) 

-0.044 (0.023) 

-0.023 (0.026) 

-0.004 (0.025) 

Lithuania 

0.422 (0.028) 

0.390 (0.030) 

0.379 (0.024) 

0.005 (0.019) 

0.003 (0.021) 

0.048 (0.024) 

Malta 

0.194 (0.021) 

0.165 (0.021) 

0.191 (0.019) 

0.003 (0.025) 

0.060 (0.020) 

0.084 (0.021) 

Morocco 

0.268 (0.057) 

0.244 (0.053) 

0.313 (0.042) 

0.097 (0.035) 

0.127 (0.032) 

0.063 (0.037) 

Northern Ireland 

0.223 (0.033) 

0.192 (0.036) 

0.193 (0.036) 

-0.067 (0.032) 

-0.049 (0.036) 

-0.072 (0.033) 

Norway 

0.390 (0.030) 

0.302 (0.032) 

0.286 (0.034) 

-0.063 (0.032) 

-0.131 (0.026) 

-0.030 (0.027) 

Oman 

0.216 (0.024) 

0.234 (0.026) 

0.219 (0.026) 

0.052 (0.016) 

0.051 (0.019) 

0.095 (0.017) 

Poland 

0.326 (0.020) 

0.241 (0.023) 

0.247 (0.020) 

-0.049 (0.019) 

-0.033 (0.021) 

0.009 (0.018) 

Portugal 

0.158 (0.034) 

0.124 (0.029) 

0.205 (0.022) 

-0.089 (0.033) 

-0.072 (0.034) 

-0.068 (0.027) 

Qatar 

0.196 (0.029) 

0.225 (0.029) 

0.211 (0.027) 

0.085 (0.024) 

0.110 (0.024) 

0.094 (0.026) 

Romania 

0.238 (0.041) 

0.219 (0.034) 

0.212 (0.030) 

-0.079 (0.026) 

-0.110 (0.024) 

-0.084 (0.024) 

Russian Federation 

0.295 (0.028) 

0.278 (0.028) 

0.284 (0.029) 

-0.025 (0.026) 

-0.030 (0.027) 

0.008 (0.022) 

Saudi Arabia 

0.149 (0.038) 

0.143 (0.040) 

0.179 (0.037) 

0.010 (0.028) 

-0.028 (0.028) 

0.008 (0.023) 

Singapore 

0.364 (0.020) 

0.341 (0.018) 

0.325 (0.019) 

-0.027 (0.018) 

0.009 (0.018) 

0.034 (0.016) 

Slovak Republic 

0.164 (0.033) 

0.133 (0.034) 

0.162 (0.029) 

-0.043 (0.030) 

-0.045 (0.035) 

-0.039 (0.026) 

Slovenia 

0.340 (0.020) 

0.201 (0.025) 

0.243 (0.020) 

-0.017 (0.025) 

0.011 (0.024) 

0.011 (0.024) 

Spain 

0.291 (0.023) 

0.258 (0.027) 

0.253 (0.024) 

0.040 (0.020) 

0.052 (0.031) 

0.097 (0.020) 

Sweden 

0.383 (0.022) 

0.205 (0.027) 

0.235 (0.021) 

0.012 (0.034) 

0.037 (0.026) 

0.053 (0.027) 

United Arab Emirates 

0.110 (0.014) 

0.119 (0.015) 

0.099 (0.014) 

0.083 (0.015) 

0.096 (0.015) 

0.111 (0.015) 

Sixth Grade Countries 

Botswana 

0.190 (0.031) 

0.216 (0.026) 

0.202 (0.025) 

-0.005 (0.026) 

-0.026 (0.023) 

-0.006 (0.021) 

Honduras 

0.168 (0.040) 

0.185 (0.036) 

0.206 (0.036) 

-0.079 (0.038) 

-0.102 (0.035) 

-0.070 (0.029) 

Benchmarking Participants 

Quebec, Canada 

0.240 (0.026) 

0.202 (0.023) 

0.201 (0.023) 

-0.048 (0.028) 

-0.010 (0.027) 

-0.020 (0.025) 

Abu Dhabi, UAE 

0.162 (0.028) 

0.172 (0.027) 

0.150 (0.027) 

0.103 (0.023) 

0.110 (0.021) 

0.117 (0.023) 

Dubai, UAE 

0.086 (0.017) 

0.085 (0.020) 

0.068 (0.018) 

0.081 (0.021) 

0.108 (0.022) 

0.111 (0.023) 


International Avg. 

0.264 (0.028) 

0.210 (0.029) 

0.218 (0.027) 

0.039 (0.025) 

0.057 (0.026) 

0.051 (0.024) 

International Std. Dev. 

0.098 (0.009) 

0.083 (0.007) 

0.077 (0.007) 

0.035 (0.005) 

0.041 (0.005) 

0.039 (0.005) 


( ) Standard errors appear in parentheses. 
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Discussion of Overall Results from the Country-by-Country Analysis 
This analysis of country differences in direct effects in the path model 
demonstrates both similarities and differences. We can first conclude that, for all 
countries, the links were supported in the hypothesized Main Path from Books 
to Activity, to Ability, and finally to achievement at the fourth grade. Given that 
there also was a strong direct effect of Parental Education on Books, this Main 
Path mediates a part of the total effect of Parental Education on achievement. 
However, there was no general effect of Gender on Books, even though there 
was a weak tendency for the parents to report more books in the home for girls; 
therefore, the Main Path does not mediate much of the Gender differences in 
achievement. 

Unexpectedly, NumLitAct had a direct effect on Ability in all countries. 
This effect implies that, in homes that place greater emphasis on literacy 
activities than numeracy activities, the child will develop a greater general 
Ability to do both numeracy and literacy tasks by the beginning of primary 
school. One possible explanation of this finding is that literacy activities have 
a broader range of influence, so that they positively impact both literacy and 
numeracy skills. A partially different interpretation is that numeracy skills at the 
beginning of primary school tend to involve both reading and writing, because 
expression of numeracy skills often is accomplished via literacy skills. 

Another possible interpretation is that this relationship is due to the fact 
that the Ability variable has a bias towards literacy skills. From the presentation 
of the measurement model in the Common model, it will be remembered that 
the two indicators of literacy skills had higher loadings on the latent Ability 
variable than had the two indicators of numeracy skills. This suggests that the 
Ability variable could be biased in the literacy direction, which would cause the 
positive relationship with the NumLitAct variable. However, even though this 
line of argument may be valid, it must be asked why the literacy tasks have such 
a strong relationship to the Ability variable. One interpretation of this would 
be that the literacy skills indicators are better indicators of a general Ability to 
perform school-related tasks than are the numeracy skills indicators. The fact 
that the latent Ability variable had relatively high relations with all of the fourth 
grade student achievement measures supports this line of reasoning. Thus, if 
the latent Ability variable is to have the desirable property of predicting school 
achievement, it may need to have an emphasis on literacy skills rather than on 
numeracy skills. 
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It also is interesting to note that, in nearly all countries, the parents 
reported a stronger emphasis on literacy activities than on numeracy activities 
when the child was a girl than when the child was a boy This suggests that 
the direct effect of NumLitAct on Ability is an important mediator of gender 
differences in achievement. For some countries there also were either positive 
or negative effects of Parental Education on NumLitAct, which implies that this 
link may mediate a part of the total effect of Parental Education on achievement 
as well. For the majority of countries, there also was a positive effect of Books 
on NumLitAct. 

Except for the Main Path and the NumLitAct- Ability link, these analyses 
did not identify other possible mediating paths that hold across countries. 
However, considerable variability across countries was observed in almost each 
and every relationship within the model. To more clearly see and interpret these 
relationships, it seems necessary to investigate the full set of relationships for 
each country. Thus, the next section reports results from country-by-country 
analyses based on the path diagrams. 

Country Results 

This section presents more detailed information about the pattern of results for 
each participating country. For each country, one path diagram displays effects 
of Parental Education on achievement, and another path diagram shows effects 
of Gender on achievement. 

For each country, a saturated model was fitted, which included all 
relationships from a particular variable to all variables to the right of it in 
the path diagram. However, the path diagrams presented here only report 
relationships that are significant at the .05 level. 

Exhibit 4.17 presents results from the statistical goodness-of-fit test and 
from the indices RMSEA, CFI and SRMR. As may be seen the model fits the 
data from all countries excellently. 

In the following section, results are presented individually for each country 
and benchmarking entity (except for Ireland, where data is missing on some 
variables) in the form of one path diagram for effects of Parental Education and 
another path diagram for effects of Gender. Brief interpretive comments relative 
to each path diagram are included. 
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Exhibit 4.17: 


Tests and Indices of Model Fit 


TIMSS fr PIRLS 
2011 


^.th 

Grade 


Country 

Chi-2 

df 

CFI 

RMSEA 

SRMR 

N 

Australia 

286.45 

50 

0.990 

0.028 

0.021 

5943 

Austria 

884.30 

50 

0.971 

0.060 

0.041 

4587 

Azerbaijan 

144.81 

50 

0.994 

0.020 

0.017 

4871 

Chinese Taipei 

154.90 

50 

0.997 

0.022 

0.011 

4265 

Croatia 

359.65 

50 

0.990 

0.037 

0.018 

4545 

Czech Republic 

247.59 

50 

0.990 

0.030 

0.021 

4433 

Finland 

263.98 

50 

0.990 

0.031 

0.019 

4541 

Georgia 

167.52 

50 

0.996 

0.022 

0.013 

4774 

Germany 

749.95 

50 

0.976 

0.060 

0.042 

3928 

Hong Kong SAR 

129.41 

50 

0.997 

0.020 

0.010 

3802 

Hungary 

437.59 

50 

0.988 

0.039 

0.028 

5149 

Iran, Islamic Rep. of 

261.64 

50 

0.995 

0.027 

0.015 

5734 

Ireland 

278.87 

50 

0.987 

0.032 

0.024 

4383 

Italy 

160.54 

50 

0.996 

0.023 

0.014 

4125 

Lithuania 

189.57 

50 

0.995 

0.025 

0.015 

4584 

Malta 

338.41 

50 

0.991 

0.041 

0.022 

3492 

Morocco 

166.95 

50 

0.995 

0.018 

0.014 

7614 

Northern Ireland 

201.95 

50 

0.990 

0.030 

0.022 

3467 

Norway 

319.07 

50 

0.984 

0.042 

0.024 

3054 

Oman 

281.29 

50 

0.994 

0.021 

0.013 

10237 

Poland 

351.74 

50 

0.991 

0.035 

0.019 

4962 

Portugal 

236.96 

50 

0.991 

0.031 

0.014 

3991 

Qatar 

171.90 

50 

0.994 

0.024 

0.014 

4104 

Romania 

123.81 

50 

0.998 

0.018 

0.008 

4643 

Russian Federation 

205.16 

50 

0.991 

0.026 

0.014 

4450 

Saudi Arabia 

149.22 

50 

0.994 

0.021 

0.013 

4470 

Singapore 

249.72 

50 

0.997 

0.025 

0.012 

6208 

Slovak Republic 

256.38 

50 

0.994 

0.027 

0.016 

5561 

Slovenia 

432.05 

50 

0.986 

0.042 

0.027 

4433 

Spain 

183.76 

50 

0.994 

0.026 

0.013 

4105 

Sweden 

457.37 

50 

0.981 

0.043 

0.024 

4482 

United Arab Emirates 

473.34 

50 

0.995 

0.024 

0.013 

14377 

Sixth Grade Countries 

Botswana 

139.52 

50 

0.998 

0.021 

0.016 

4165 

Honduras 

136.57 

50 

0.995 

0.021 

0.023 

3830 

Benchmarking Participants 

Quebec, Canada 

270.68 

50 

0.989 

0.033 

0.019 

4142 

Abu Dhabi, UAE 

177.27 

50 

0.996 

0.025 

0.014 

4100 

Dubai, UAE 

280.46 

50 

0.996 

0.028 

0.015 

5922 


Reported values are means over analyses of five plausible values. 
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Summary of Country Results 

The path diagrams for the TIMSS and PIRLS 2011 assessment participants 
indicate both similarities and differences across countries. The similarities 
manifest themselves in several different ways. For nearly all countries, the 
effects of Parental Education on achievement were mediated via Parental 
Education, Books, Activities, and Abilities, or what we have termed the Main 
Path. In addition, for most countries, there also were substantial effects of 
Books on achievement. Furthermore, for most of the countries, we observed a 
pattern where a stronger emphasis on literacy than on numeracy activities was 
associated with a higher level of Ability, which in turn had positive effects on 
achievement in all domains. 

The differences also manifested themselves in several ways. One major 
source of differences was the strength of the estimated direct effects, and 
therefore also the size of the indirect effects. Such differences maybe reflections 
of real differences in strength of relationships between variables; however, they 
also may be due to issues of measurement, such as floor and ceiling effects. 
Another source of differences was that the estimated coefficients sometimes had 
different signs in different countries, such as the relationship between Parental 
Education and the NumLitAct variable. Differences in sign of relationship seem 
more likely to reflect substantive differences than problems of measurement. 

The similarities and differences combine in such a way as to make the 
pattern of relationships appear markedly different from one country to another, 
in spite of the fact that it also is possible to recognize the three basic patterns 
of mediating relationships between Parental Education and achievement, 
described above. 

For Gender, both the total effects and the mediating mechanisms were 
quite different across the achievement domains. Even though there were effects 
for mathematics and science, these tended to vary from country to country. For 
reading, though, the general pattern was one of an achievement advantage for 
girls, and in many countries it was possible to explain this in terms of a stronger 
emphasis on literacy activities than on numeracy activities for girls. 

Discussion of the Empirical Findings 

First, it may be noted that the estimates of the standardized regression 
coefficients that we obtained with the Common model for pooled data are very 
close to the means of the standardized parameter estimates for the individual 
countries. The Common model clearly brings out how the effect of Parental 
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Education on achievement follows the Main Path, i.e., that it is mediated via 
Books, Activities, and Abilities. The Common model also showed how Books 
mediates Parental Education through its direct effects on the three achievement 
measures. Furthermore, the Common model identified the effect of a stronger 
emphasis on literacy activities than numeracy abilities on Ability, which in its 
turn had effects on the three domains of achievement. However, according to 
the Common model, this mechanism did not mediate the effect of Parental 
Education on achievement, except for a trivially small effect via Books. This was 
because of heterogeneity in the relationship between Parental Education and 
NumLitAb across the countries. 

However, for Gender, the Common model identified a stronger emphasis 
on literacy activities than on numeracy activities for girls as the first part of 
a mechanism that accounts for gender differences in reading achievement. 
The second part of this mechanism is that the emphasis on literacy activities 
influences the ability to perform both literacy and numeracy tasks at the age 
children begin primary school. In the Common model there were no other 
mediating relations between Gender and achievement. 

Given that this study did not predict the effect on Ability of a stronger 
emphasis on literacy than numeracy activities, it may be asked whether this is 
a dependable phenomenon and how it should be interpreted. The fact that this 
pattern of relationships has been identified in practically every country indicates 
a high degree of empirical consistency. When the measurement model for the 
Common model for pooled data was estimated, it was observed that the literacy 
skills measures had higher loadings on the Ability factor than had the numeracy 
skills measures. This could be taken as an indication that Ability is biased in 
favor of literacy, which could explain the positive relationship with NumLitAct. 
However, another interpretation of this finding is that literacy skills are more 
generally applicable than are numeracy skills. For example, numeracy tasks are 
often presented in written form and require written responses. 

Previous research reviewed in the Introduction supports the view that 
numeracy activities and tasks tend to be subordinate to literacy activities and 
tasks. Thus, there are few, if any, interventions which focus more exclusively 
on development of numeracy, and it has been hypothesized that numeracy 
skills develop as a function of training in language and problem-solving skills 
at young age (Doig, McCrae, & Rowe, 2003). Interestingly enough, Anders et 
al. (2012) found that numeracy skills were more highly related to the quality 
of the home learning environment with respect to literacy than with respect 
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to numeracy. These authors also observed that numeracy activities tend to be 
less frequent than literacy activities, which both makes it difficult to measure 
numeracy activities reliably, and to identify their effects on development of 
skills. Even though further research is needed to clarify the nature of literacy and 
numeracy skills, it does seem that both theory and previous empirical research 
support the idea that literacy activities can influence the development of both 
literacy and numeracy skills. 

Effects of Parental Education 

For practically all countries, support was obtained for the Main Path, or the 
sequence Parental Education, Books, Activity, Ability, and fourth grade student 
achievement in mathematics, science, and reading. The high degree of generality 
of this mechanism across countries is an interesting phenomenon, which has 
been observed many times before, and for which at least intuitive explanations 
have been offered. We will not pursue theoretical discussions here, but it can be 
noted that even though all links in the Main Path are significant they do vary in 
strength across countries. This variation can be due to issues of measurement, 
such as floor and ceiling effects in certain countries, and it also can have 
substantive grounds. It is an important task for further research to investigate 
different sources of variation across countries more closely. 

In addition to the indirect effect via the Main Path, the Books variable 
also had strong direct effects on all three achievement variables, though with 
large variation across countries. To account for these direct effects we need 
other hypothesized mechanisms and further mediating variables. In previous 
research many such hypotheses have been investigated, and the results show, 
among other things, that parental expectations and the parents’ function as 
role models are important mediating mechanisms to account for the effects of 
Parental Education on achievement. It is reasonable to expect that the number 
of books in the home can play an important part in such mediating mechanisms. 

We also have identified another path between Parental Education and 
achievement, which was unexpected, but which is present in the majority of 
the countries. The core of this path is the relationship between NumLitAct and 
Ability, which implies that in homes where there is stronger emphasis on literacy 
activities than numeracy activities there is a positive effect on the ability to 
perform both numeracy and literacy tasks at the beginning of primary school. 
In some countries, there was a direct effect (positive or a negative) of Parental 
Education on NumLitAct, while in other countries, there was an indirect effect 
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via Books. There also were countries in which both the direct and the indirect 
effect could be observed, and some countries in which there was neither a direct, 
nor an indirect effect of Parental Education on NumLitAct. 

A positive effect of Parental Education on NumLitAct was observed for 
seven countries, while a negative effect was observed for four countries. Thus, 
in the former group of countries, parents with a higher level of education placed 
more emphasis on literacy activities than on numeracy activities, while in the 
latter group these parents placed more emphasis on numeracy activities than 
on literacy activities. However, within both groups of countries, the direct effect 
of NumLitAct on Ability was positive and approximately the same size (0.23 
vs. 0.18). This indicates that the mechanism of influence of the two types of 
activities on ability is invariant across these two categories of countries. 

All three participating Nordic countries (Finland, Norway, and Sweden) 
showed a positive effect of Parental Education on NumLitAct, along with the 
Czech Republic, Morocco, Saudi Arabia, and Slovenia. In the Nordic countries 
there is a long tradition of literacy and reading aloud for the children is a 
common practice, particularly among parents with a higher level of education. 

One hypothesis to account for the different directions of the effect of 
Parental Education on NumLitAct may be that, in certain countries, higher 
education places more emphasis on fields such as technology and science, 
while in other countries there is more emphasis on letters and arts. The four 
countries with a negative effect of Parental Education on NumLitAct (implying 
greater emphasis on numeracy than on literacy activities) were Chinese Taipei, 
Honduras, Iran, and Oman; for at least some of these, higher education may be 
more oriented towards technology and science. 

The direct effect of Books on NumLitAct was significant and positive for 
around 20 countries. Thus, availability of books was a mediator for activities 
oriented towards literacy. 

It also is interesting to observe that, in many cases, there was not only a 
positive indirect effect of NumLitAct on the three achievement variables via 
Ability, but there were also direct effects of NumLitAct on the achievement 
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variables. A negative effect on mathematics achievement was the most 
frequently observed outcome, but in some cases there were negative effects on 
mathematics and science, and in other cases a positive effect on reading. The 
combination of the indirect effects and these direct effects creates an uneven 
profile of achievement, with a relatively higher level of achievement in reading 
than in mathematics. 

There also were some other frequently recurring patterns of relations in 
the path model. Thus, the NumLitAb variable, which indicates whether literacy 
skills at school start are rated higher than numeracy skills or vice versa, had for 
many countries a negative influence from Books, and it often also had negative 
direct effects on achievement. For many countries there also was a direct 
effect NumLitAct on NumLitAb. However, these effects tended to be weak and 
sometimes difficult to interpret. 

Effects of Gender 

As previously discussed, only in reading was a strong total effect for Gender 
observed, and only in reading was the study able to identify mediating 
mechanisms that were reasonably consistent across countries. The most 
powerful path accounting for gender differences in reading achievement was 
the direct effect of NumLitAct on Ability, which in turn had positive direct 
effects on achievement. Here, too, a negative direct effect of NumLitAct on 
mathematics achievement was frequently observed. 

For any mediating effect to occur, it is, of course, also necessary that there 
be a Gender effect on NumLitAct, such that greater emphasis on literacy than on 
numeracy activities is reported for girls. This effect was positive and significant 
in all but two countries (Morocco and Saudi Arabia). The participants with the 
strongest direct effect of gender on NumLitAct were Norway, Lithuania, Sweden, 
the Canadian province of Quebec, Germany, Finland, and Poland. It is quite 
interesting to observe such strong differentiation of activities between boys and 
girls in societies that emphasize gender equality, particularly the three Nordic 
countries. One tentative explanation for this may be that the parents, following 
strong literacy traditions, offer book-sharing activities with their children, but 


EFFECTS OF HOME BACKGROUND ON STUDENT ACHIEVEMENT IN 
TIMSS & PIRLS READING, MATHEMATICS, AND SCIENCE AT THE FOURTH GRADE 

International Study Center 

Lynch School of Education, Boston College v H A r I fcK 4 


245 



that boys are less interested in participating in such activities than girls because 
of their perceptions of what are appropriate activities for girls and boys. 

Another mediating path went through NumLitAb, on which Gender 
had a positive effect in most countries. This positive effect implies that girls 
were assessed as being relatively better at doing literacy tasks than at doing 
numeracy tasks at the beginning of primary school. The NumLitAb variable 
was positively related to reading in many countries, or to reading and science 
in other countries, or negatively related to mathematics in yet other countries. 

Limitations and Future Research 

While this study has identified several interesting mediating mechanisms that 
at least partly explain the effects of Parental Education and Gender on fourth 
grade student achievement, it also generated many questions that need to be 
addressed in future research. Some of these questions are discussed below. 

From a statistical point of view, the indirect effects identified in many of 
the models are quite small, and it may be asked whether they are large enough 
to warrant any strong conclusions. However, it must be remembered that the 
information about the mediating variables in the model is based on a limited 
number of responses to questionnaire items. Further, given the well-known 
problems of reliability and validity of such information, it is surprising that it 
has been possible to identify so many consistent and meaningful patterns of 
relationships among the variables. Yet, because of the problems of measurement, 
it is likely that the strength of the relationships among the variables is 
underestimated, which in turn causes the indirect effects to be underestimated. 

A modeling approach with latent variables was used to at least partially 
address the measurement problems. However, for practical reasons a “testlet” 
approach was adopted, and this approach does not fully utilize the information 
available in the item responses. It would therefore be useful to put further effort 
into the development of the measurement model, through using item level data 
rather than testlet data. 

Yet another measurement problem is that the parents provided the 
information retrospectively; retrospective information tends to be unreliable, 
and there also is risk for systematic bias caused by selective memory and 
reinterpretation of earlier events in light of later events and developments. It 
is difficult to assess to what extent such threats to the validity of the Home 
Questionnaire data are present in the current data. However, the fact that the 
assessment is low-stakes implies that no gains are made from misreporting of 
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facts. A more optimal approach for collecting information about the quality of 
home learning environments for literacy and numeracy was used by Anders et 
al. (2012), who relied on a combination of self-report questionnaires, interviews, 
and observations. While such data-collection techniques cannot be used in 
large-scale international assessments, they may be useful for investigating the 
measurement characteristics of questionnaire data and for optimizing the design 
of questionnaire items. 

It also should be acknowledged that there is further information in the 
TIMSS and PIRLS 2011 data that could be used to extend the model. For 
example, there are data on parental attitudes towards reading and on parental 
reading practices, both of which maybe hypothesized to influence actual use of 
books. The model also could be extended with more student variables, reflecting, 
for example, attitudes towards reading, reading practices, and computer use. 
Thus, there are many possibilities to extend the current study in different 
directions. 

Conclusion 

The aims of this study were twofold: to investigate the extent to which 
parental education and gender influence fourth grade student achievement 
in reading, mathematics, and science in different countries; and to investigate 
the mechanisms through which parental education and gender influence 
achievement in these three core subjects via books in the home, literacy and 
numeracy activities, and the child’s ability to carry out literacy and numeracy 
tasks when starting school. Through applying path modeling techniques to data 
from the TIMSS and PIRLS 2011 assessments and Home Questionnaires, it had 
been possible to identify some important mechanisms through which Parental 
Education and Gender influence achievement in mathematics, science, and 
reading at the fourth grade. 

For nearly all countries, the effects of Parental Education on achievement 
were mediated via Parental Education, Books, Activities, and Abilities, or 
what we have termed the Main Path. According to this mechanism, parental 
education influences the number of books available in the home. In turn, the 
number of books is related to the frequency of home activities oriented towards 
both literacy and numeracy, and these activities influence the general level 
of literacy and numeracy skills that the child has developed upon beginning 
primary school. The literacy and numeracy skills that the child brings to school 
influence achievement at the fourth grade. In addition, for most countries, 
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there also were substantial direct effects of number of books in the home on 
achievement. 

Another mechanism is that a stronger emphasis on literacy than on 
numeracy activities influences the general level of literacy and numeracy skills 
children have developed by the time they begin primary school, and this in turn 
influences achievement at the fourth grade. It is more common for girls than 
for boys to have such an emphasis, which partially explains the higher level of 
reading achievement for girls. In homes with a larger number of books there 
is in many countries also a tendency to put more emphasis on literacy than on 
numeracy activities, which influences the general level of numeracy and literacy 
skills at school start, which influences achievement. 

For reading, though, the general pattern was one of an achievement 
advantage for girls. 

In terms of gender differences, both the total effects and the mediating 
mechanisms were quite different across the achievement domains. Even though 
there were effects for mathematics and science, these tended to vary from 
country to country, and so it is difficult to generalize the effects for mathematics 
and science across countries. However, for reading, the general pattern was one 
of an achievement advantage for girls, and in many countries it was possible 
to explain this in terms of a stronger emphasis on literacy activities than on 
numeracy activities for girls. 

While the abovementioned mechanisms could be identified in almost all 
of the 37 participating countries and benchmark entities, interesting differences 
among the countries also could be identified, both with respect to the strength 
of estimated relationships, and in the patterns of relationships among variables. 

The research presented in this chapter can be extended in many different 
ways in order to obtain better estimates of the relationships in the model, as well 
as to allow investigations of further variables and hypothesized mechanisms. 
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Exhibit 4.1 8: Australia 


TIMSS &PIRLSI 
201ll 


PARENTAL EDUCATION The total effects were .33, .35, and .33 for mathematics, science, and 
reading, respectively, and the corresponding total indirect effects were .11, .14, and .12. Most 
of the indirect effect was mediated via Books and via the Main Path. However, the number 
of books in the home also was related to more literacy than numeracy activity, which in turn 
influenced Ability positively, and Ability had significant direct effects on achievement in all 
three domains. There also were weak indirect effects via NumLitAb. 



GENDER The total effects were -.03, .00, and .12 for mathematics, science, and reading, 
respectively. The total indirect effects were all close to 0. Most of the Gender effect on reading 
was direct. However, Gender was related to an overrepresentation of literacy activity, which 
had a positive effect on achievement in all three domains via Ability. Gender also was related 
to a relatively higher assessment of literacy skills than numeracy skills, which was negatively 
related to achievement in all three domains. 
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SOURCE: lEA's Trends in International Mathematics and Science Study and Progress in International Reading Literacy Study - TIMSS and PIRLS 201 1 




Exhibit 4.19 Austria 


TIMSS &PIRLSI 
201ll 


PARENTAL EDUCATION The total effects were .31, .33, and .32 for mathematics, science, and 
reading, respectively, and the corresponding total indirect effects were .22, .23, and .22. Thus, 
a substantial proportion of the total effect was indirect. Most of the indirect effects were 
mediated via Books and via the Main Path. However, the number of books in the home also 
was associated with higher assessed numeracy skills than literacy skills, which in turn was 
associated with achievement in mathematics and science. 



GENDER The total effects of Gender were -.07, -.09, and .06 for mathematics, science, and | 
reading, respectively, and the corresponding total indirect effects were -.02, -.01, and .01. There | 
was more literacy than numeracy activity for girls, which affected achievement positively | 

O 

via Ability, and mathematics achievement negatively through a direct effect. There also were 1/1 
higher ratings of literacy than numeracy skills for girls, which was associated with a lower 
level of performance in mathematics and science. 
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Exhibit 4.20 Azerbaijan 


TIMSS &PIRLSI 
201ll 


PARENTAL EDUCATION The total effects were .11, .14, and .15 for mathematics, science, and 
reading, respectively, and the corresponding total indirect effects were .05, .06, and .03. Thus, 
the total effects were among the lowest observed. However, for mathematics and science a 
rather large proportion of the total effect was mediated. These indirect effects were mediated 
via Books and via Books and NumLitAb. The Books variable predicted a higher ability to 
do numerical tasks than literacy tasks and the NumTitAb variable was negatively related to 
mathematics and science achievement, implying positive effects of having higher ability to 
do numerical tasks. 



GENDER The total effects were .04, .03, and .10 for mathematics, science, and reading, 
respectively. The total indirect effects were all close to 0. However, there were significant 
negative indirect effects of Gender on mathematics and science. This was because Gender 
predicted more literacy than numeracy activity, which in turn was positively related to the 
NumLitAb variable, which was negatively related to mathematics and science achievement. 
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SOURCE: lEA's Trends in International Mathematics and Science Study and Progress in International Reading Literacy Study - TIMSS and PIRLS 201 1 




Exhibit 4.21 : Chinese Taipei 


TIMSS fr PIRLS 
2011 


^th 

Grade 


PARENTAL EDUCATION The total effects of Parental Education were .37, .39, and .34 for 
mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were .17, .17, and .13. The indirect effects were mediated via the Main Path and via Books. 
However, there also was a direct effect of Parental Education on Activity. Parental Education 
also was associated with more numeracy than literacy activity, which influenced mathematics 
and science achievement positively. 



GENDER The total effects of Gender were .01, -.05, and .10 for mathematics, science, and 
reading, respectively, and the corresponding total indirect effects were .02, .02, and .03. Thus, 
girls outperformed boys in reading, while boys had higher achievement in science. There 
was a positive indirect effect via Ability on achievement for girls in all domains. For girls, 
there was more of an emphasis on literacy activities than on numeracy activities. This had a 
positive effect on achievement in all three domains via Ability, and negative direct effects on 
mathematics and science achievement for girls. 
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Exhibit 4.22: Croatia 


TIMSS & PIRLS 
2011 


^.th 

Grade 


PARENTAL EDUCATION The total effects of Parental Education were .31, .32, and .31 for 
mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were .14, .15, and .14. The indirect effects were mediated via the Main Path and via Books. 
The number of books in the home also was related to an overrepresentation of literacy activity, 
which influenced achievement via Ability, and there also was a positive effect of Books on 
mathematics achievement via NumLitAb. 



GENDER The total effects of Gender were -.08, -.04, and .12 for mathematics, science, and 
reading, respectively, and the corresponding total indirect effects were .01, .02, and .04. Girls 
thus outperformed boys in reading, while boys had higher achievement in mathematics and 
science. For girls there was an overrepresentation of literacy activities, which had a positive 
indirect effect on achievement in all domains via Ability, but also negative direct effects on 
mathematics and science achievement. Girls furthermore had higher rated literacy skills than 
numeracy skills, which was negatively related to mathematics achievement. 
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SOURCE: lEA's Trends in International Mathematics and Science Study and Progress in International Reading Literacy Study - TIMSS and PIRLS 201 1 




Exhibit 4.23: Czech Republic 


TIMSS & PIRLS 
2011 


^th 

Grade 


PARENTAL EDUCATION The total effects of Parental Education were .31, .29, and .29 for 
mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were .14, .16, and .16. The indirect effect was to a large extent mediated via the Main Path and 
via Books. Both Parental Education and Books also were related to a stronger emphasis on 
literacy activity than on numeracy activity, which had positive effects on achievement in all 
three domains via Ability Books and Activity also were related to higher assessed numeracy 
skills than literacy skills. 



GENDER The total effects of Gender were -.08, -.09, and .06 for mathematics, science, and 
reading, respectively and the corresponding total indirect effects were .01, .01 and .01. Girls 
thus outperformed boys in reading, while boys had higher achievement in Mathematics and 
science. The advantage for girls in reading was, to a small extent, mediated via activities that 
emphasized literacy more than numeracy for girls, and the effect of which was mediated via 
Ability. The NumLitAb variable also mediated effects from Activity and NumLitAct. 
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Exhibit 4.24: Finland 


TIMSS & PIRLS 
2011 


^.th 

Grade 


PARENTAL EDUCATION The total effects of Parental Education were .29, .28, and .28 for 
mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were . 13, . 15, and .16. The indirect effect was to a large extent mediated via the Main Path and 
via Books. Parental Education influenced mathematics achievement negatively via activities 
that emphasized literacy more than numeracy, while this influenced achievement in all 
domains positively via Ability. 



GENDER The total effects of Gender were -.06, -.01, and .17 for mathematics, science, and 
reading, respectively, and the corresponding total indirect effects were .01, .03, and .06. Girls 
thus outperformed boys in reading by a wide margin, while boys had higher achievement in 
mathematics. For girls, literacy activities were more emphasized than numeracy activities, 
which had a positive indirect effect on achievement in all three domains via Ability. Elowever, 
this emphasis also had a negative direct effect on mathematics achievement for girls. 
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SOURCE: lEA's Trends in International Mathematics and Science Study and Progress in International Reading Literacy Study - TIMSS and PIRLS 201 1 





Exhibit 4.25: Georgia 


TIMSS & PIRLS 
2011 


^th 

Grade 


PARENTAL EDUCATION The total effects of Parental Education were .28, .29, and .31 for 
mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were .13, .14, and .14. The indirect effects were to a very large extent mediated via Books and 
via the Main Path. The number of books in the home also was associated with a relatively 
stronger emphasis on numeracy activities than literacy activities, which had a positive effect 
on mathematics achievement and negative effects on achievement in all three domains via 
Ability. Similar effects of Books were mediated via NumLitAb. 



GENDER The total effects of Gender were .04, .06, and . 15 for mathematics, science, and reading, | 
respectively, and the corresponding total indirect effects were .00, .01, and .01. Girls thus | 
outperformed boys in all three domains of achievement, and particularly so for reading. For | 

O 

girls, there was a relatively stronger emphasis on literacy activities than numeracy activities, 1/1 
which influenced achievement in all three domains positively, via Ability. This pattern of 
activities also was negatively related to mathematics achievement. Girls also were assessed 
higher in literacy skills than in numeracy skills, which was negatively related to achievement. 
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Exhibit 4.26: Germany 


TIMSS & PIRLS 
2011 


^.th 

Grade 


PARENTAL EDUCATION The total effects of Parental Education were .36, .38, and .36 for 
mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were .19, .23, and .21. The total indirect effect was, to a large extent, mediated via Books and 
via the Main Path. The number of books in the home also was associated with a relatively 
stronger emphasis on literacy activities than numeracy activities, which indirectly affected 
achievement positively via Ability. To some extent the effect of Parental Education was 
mediated via the balance of the assessment of literacy and numeracy skills, as well as more 
highly educated parents tending to assess numeracy skills higher, both directly and mediated 
via Books. 



GENDER The total effects of Gender were -.07, -.09, and .06 for mathematics, science, and 
reading, respectively, and the corresponding total indirect effects were -.02, .00, and .01. Girls 
thus outperformed boys in reading, while boys had higher achievement in mathematics and 
science. For girls, there was a stronger emphasis on literacy activities than numeracy activities, 
which had a positive effect on achievement, via Ability. However, this imbalance also was 
negatively related to mathematics achievement. Girls also were assessed as having stronger 
literacy skills than numeracy skills, which was negatively related to achievement. 
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Exhibit 4.27: Hong Kong SAR 


TIMSS & PIRLS 
2011 


^th 

Grade 


PARENTAL EDUCATION The total effects of Parental Education were .16, .15, and .12 for 
mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were .09, .10, and .08. Thus, there was only a small total effect of Parental Education on the 
three achievement measures, and particularly so for reading. However, the total indirect effect 
accounted for a considerable part of the total effect. Indirect effects went via the Main Path 
and via Books. 



GENDER The total effects of Gender were -.05, -.07, and .13 for mathematics, science, and | 

c 

reading, respectively, and the corresponding total indirect effects were .03, .04, and .04. Girls | 
thus outperformed boys in reading, while boys had higher achievement in mathematics | 
and science. For girls, there was a stronger emphasis on literacy activities than numeracy | 
activities, which had a positive indirect on achievement in all domains via Ability. Girls also | 

O 

were assessed somewhat higher on Ability than were boys. 
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Exhibit 4.28: Hungary 


TIMSS & PIRLS 
2011 


^.th 

Grade 


PARENTAL EDUCATION The total effects of Parental Education were .55, .55, and .53 for 
mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were .28, .29, and .24. There was thus a substantial total effect of Parental Education on the 
three achievement measures, and the total indirect effect accounted for a considerable part of 
the total effect. Indirect effects went via the Main Path and via Books. Parents with a higher 
level of education tended to assess numeracy skills higher than literacy skills, which had a 
positive effect on achievement in all three domains. Similar effects of Parental Education on 
both NumLitAct and NumLitAb were mediated via Books. 



GENDER The total effects of Gender were -.03, -.03, and .10 for mathematics, science, and 
reading, respectively, and the corresponding total indirect effects were -.02, -.01, and .00. Girls 
thus outperformed boys in reading, while boys had higher achievement in mathematics and 
science. For girls, there was more of an emphasis on literacy activities than numeracy activities. 
This was associated with a positive indirect effect on achievement in all three domains via 
Ability, at the same time as there were negative effects on achievement, and particularly so in 
mathematics and science. 
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Exhibit 4.29: Iran, Islamic Republic of 


TIMSS fr PIRLS 
2011 
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Grade 


PARENTAL EDUCATION The total effects of Parental Education were .44, .45, and .43 for 
mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were .19, .20, and .18. Indirect effects went particularly via the Main Path and via Books. 
Parents with a higher level of education tended to place higher emphasis on numeracy activities 
than on literacy activities, which had a negative effect on achievement in all three domains via 
Ability. However, there also was a positive direct effect on mathematics achievement. Parental 
Education and Books also both were associated with assessing numeracy skills higher than 
literacy skills, which was positively associated with achievement. 



GENDER The total effects of Gender were -.02, -.03, and .11 for mathematics, science, and | 

O 

reading, respectively, and the corresponding total indirect effects were -.01, -.01, and -.01. 1/1 
There was a small positive indirect effect via the Main Path on achievement for girls in all 
domains. For girls, there was a stronger emphasis on literacy activities than on numeracy 
activities, and via Ability this had a positive effect on achievement in all domains. However, 
there also were strong negative direct effects on achievement in all three domains of this 
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Exhibit 4.30: Italy 
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PARENTAL EDUCATION The total effects of Parental Education were .24, .28, and .30 for 
Mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were .09, .14, and .14. The indirect effects went via the Main Path, and there was also an 
effect of Books such that, with more books in the home, there was a stronger emphasis on 
literacy activities than on numeracy activities. This literacy emphasis had a positive effect on 
achievement in all three domains, which was mediated via Ability. 



GENDER The total effects of Gender were -.06, -.05, and .03 for mathematics, science, and 
reading, respectively, and the corresponding total indirect effects were .00, .01, and .02. For 
girls, there was more of an emphasis on literacy than on numeracy activities, which had an 
indirect effect on achievement via Ability. 
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Exhibit 4.31 : Lithuania 


TIMSS &PIRLSI 
201ll 


PARENTAL EDUCATION The total effects of Parental Education were .36, .35, and .35 for 
mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were .17, .18, and .19. Indirect effects went via the Main Path and via Books. The number 
of books in the home also was related to a stronger emphasis on literacy activities than on 
numeracy activities, which had an indirect effect on achievement via Ability. There also was 
an indirect effect of Parental Education on achievement via Ability. 
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GENDER The total effects of Gender were -.01, -.01, and .15 for mathematics, science, and 
reading, respectively, and the corresponding total indirect effects were .04, .06, and .07. Girls 
thus had a considerably higher level of achievement than boys in reading. This pattern of 
gender differences was partially mediated via a stronger emphasis on literacy activities than 
on numeracy activities, which had an indirect effect on achievement via Ability. In addition 
to the direct effect of Gender, there was an indirect effect on NumLitAct via Books in the 
same direction. There also was an effect of Gender via the Main Path on achievement in all 
domains, and an effect on reading achievement via NumLitAb. 
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Exhibit 4.32: Malta 
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PARENTAL EDUCATION The total effects of Parental Education were .34, .45, and .44 for 
mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were . 1 3, . 1 5, and .15. There was thus a very strong total effect of Parental Education on reading 
and science achievement and a strong effect on mathematics achievement. The total indirect 
effect accounted for about one-third of the total effect. Indirect effects went particularly via 
the Main Path and via Books. There also was an effect of Books, such that the variable was 
related to a higher level of assessed numeracy skills than literacy skills, which in turn had 
positive effects on science and reading achievement. 



GENDER The total effects of Gender were -.04, -.04, and .10 for mathematics, science, and 
reading, respectively, and the corresponding total indirect effects were .02, .02, and .02. The 
indirect effects were partially mediated via the Main Path and via Books. For girls, there 
also was a stronger emphasis on literacy activities than on numeracy activities, which had a 
positive effect on achievement, which was mediated via Ability. 
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Exhibit 4.33: Morocco 


TIMSS &PIRLSI 
201ll 


PARENTAL EDUCATION The total effects of Parental Education were .19, .19, and .24 for 
mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were .00, .03, and .05. The indirect effects went in particular via the Main Path and also via 
Ability, on which there were effects of Parental Education and Books. Higher levels of Parental 
Education and Books also were associated with a stronger emphasis on literacy activities than 
on numeracy activities, which influenced achievement positively in all domains via Ability, 
but which also had negative direct effects on mathematics and science achievement. A higher 
level of Activity was associated with a higher assessment of literacy skills than numeracy skills, 
which had positive effects on mathematics and science achievement. 



GENDER The total effects of Gender were .03, .04, and .13 for mathematics, science, and reading, 
respectively, and the corresponding total indirect effects were .01, .01, and .01. Girls thus had 
a higher level of achievement than boys in reading, and to a smaller extent in mathematics 
and science. A somewhat higher level of Activity was reported for girls than for boys, 
which influenced achievement positively via Ability. Activity had negative direct effects on 
achievement in all three domains, and positive indirect effects on mathematics and science 



.79 


TIMSS AND PIRLS 201 1 RELATIONSHIPS REPORT 

264 CHAPTER 4 



TIMSS & PIRLS 

International Study Center 

Lynch School of Education, Boston College 




Exhibit 4.34: Northern Ireland 
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PARENTAL EDUCATION The total effects of Parental Education were .38, .39, and .36 for 
mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were .13, .16, and .14. The indirect effects of Parental Education were mediated via the Main 
Path and via Books. In families with a larger number of books, greater emphasis was placed 
on literacy activities than on numeracy activities, which had a positive indirect effect on 
achievement via Ability. In such families, numeracy skills also were assessed higher than 
literacy skills, which was associated with a higher level of achievement in mathematics 
and reading. 



GENDER The total effects of Gender were .02, .02, and .13 for mathematics, science, and 
reading, respectively, and the corresponding total indirect effects were .03, .04, and .04. Girls 
thus outperformed boys in reading. The effect of Gender was partially mediated via Ability. 
Furthermore, for girls, there was more emphasis on literacy activities than on numeracy 
activities, which had positive indirect effects on achievement via Ability. 
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Exhibit 4.35: Norway 
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PARENTAL EDUCATION The total effects of Parental Education were .25, .28, and .26 for 
mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were .10, .17, and .16. Compared to other countries there was thus a relatively weak total 
effect of Parental Education on reading, mathematics, and science achievement, and a rather 
large part of the total effect was indirect. The indirect effects were due to the Main Path and to 
mediation via Books. Higher Parental Education also was associated with a stronger emphasis 
on literacy activities than on numeracy activities, which in turn influenced achievement 
positively via Ability. It also was associated with higher assessed literacy skills than numeracy 
skills, which had negative direct effects on mathematics and science achievement. 



GENDER The total effects of Gender were -.04, -.01, and .13 for mathematics, science, and 
reading, respectively, and the corresponding total indirect effects were .00, .03, and .06. Girls 
thus had a higher level of achievement than boys in reading, while boys had a somewhat 
higher level of achievement in mathematics. The indirect effect on reading was partially due 
to the fact that, for girls, there was a stronger emphasis on literacy activity than on numeracy 
activity. This affected achievement positively via Ability and it also had a negative direct effect 
on mathematics achievement. Girls also were assessed as having better literacy skills than 
numeracy skills, which was negatively related to mathematics and science achievement. 
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Exhibit 4.36: Oman 
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PARENTAL EDUCATION The total effects of Parental Education were .30, .31, and .32 for 
mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were .09, .08, and .09. The indirect effects were mediated via the Main Path and via Books. 
There also was a direct effect of Parental Education on Activity. For higher levels of Parental 
Education, there was a relatively stronger emphasis on numeracy than on literacy activities, 
which indirectly influenced achievement in all domains negatively via Ability. However, there 
also were positive indirect effects of Parental Education via Books, which was associated with 
more literacy activity than numeracy activity. Parents with a higher level of education also 
tended to assess literacy skills higher than numeracy skill, which was positively related to 
achievement in all three domains. 



GENDER The total effects of Gender were .13, .14, and .20 for mathematics, science, and reading, 
respectively, and the corresponding total indirect effects were .02, .02, and .03. Girls thus had 
a considerably higher level of achievement than boys in reading, and they also outperformed 
boys in mathematics and science. The indirect effects were small for all three outcomes. For 
girls, there was a relatively stronger emphasis on literacy than numeracy activities, which 
influenced achievement in all domains via Ability. Girls also were assessed relatively higher 
in literacy skills than numeracy skills, which influenced achievement in all three domains. 
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Exhibit 4.37: Poland 
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PARENTAL EDUCATION The total effects of Parental Education were .43, .44, and .43 for g 
mathematics, science, and reading, respectively, and the corresponding total indirect effects | 
were .15, .16, and .14. The indirect effects were mediated via the Main Path and via Books. J 

i/i 

In homes with many books, there also was a tendency to place relatively more emphasis p 

>N 

on literacy than on numeracy activity. This had a positive effect on Ability, which in turn 1 
influenced achievement in all three domains. 1 



GENDER The total effects of Gender were -.06, -.02, and .11 for mathematics, science, and | 

c 

reading, respectively, and the corresponding total indirect effects were .02, .04, and .04. | 
Girls thus had a considerably higher level of achievement than boys in reading, while boys | 
outperformed girls in mathematics and science. The indirect effects occurred mainly because | 
for girls there was relatively more emphasis on literacy than on numeracy activities, which | 

O 

influenced Ability positively, and which in turn had a positive effect on achievement in all 1/1 
three domains. 
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Exhibit 4.38: Portugal 
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PARENTAL EDUCATION The total effects of Parental Education were .30, .30, and .31 for 
mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were .18, .17, and .18. The relatively substantial indirect effects went via the Main Path and 
via Books. There also was a weak effect via Books, because homes with many books tended 
to assess numeracy skills higher than literacy skills, which in turn had a positive effect on 
mathematics and science performance. 



GENDER The total effects of Gender were -.04, -.03, and .11 for mathematics, science, and 
reading, respectively, and the corresponding total indirect effects were -.01, .00, and .00. Girls 
thus had a considerably higher level of achievement than boys in reading, while boys had 
a higher level of achievement than girls in mathematics and science. There was an indirect 
effect via the pattern of activities, with more emphasis on literacy than on numeracy activities 
for girls. This indirectly affected achievement positively via an effect on Ability. There also 
was a small indirect effect via the pattern of assessed skills, with girls having relatively higher 
assessed literacy skills than numeracy skills, which had negative effects on achievement in 
mathematics and reading. 
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Exhibit 4.39: Qatar 
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PARENTAL EDUCATION The total effects of Parental Education were .39, .38, and .40 for g 
mathematics, science, and reading, respectively, and the corresponding total indirect effects | 
were .09, .08, and .08. The indirect effects were mediated via the Main Path and via Books. In J 

i/i 

homes with a larger number of books, there was a relatively greater emphasis on literacy than p 

>N 

on numeracy activities, which had a positive effect on achievement in all three domains via 1 
Ability, along with a negative direct effect on mathematics achievement. | 



.83 


GENDER The total effects of Gender were .06, .11, and .14 for mathematics, science, and reading, 
respectively, and the corresponding total indirect effects were .01, .02, and .02. Girls thus had 
a considerably higher level of achievement than boys in reading and they also outperformed 
boys in mathematics and science. There was a small mediating effect via the pattern of 
activities, with a stronger emphasis on literacy than on numeracy activities for girls. This had 
a negative direct effect on mathematics achievement, and also a positive effect on Ability, 
which in turn had positive effects on achievement in all three domains. 
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Exhibit 4.40: Romania 
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PARENTAL EDUCATION The total effects of Parental Education were .43, .47, and .49 for 
mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were .22, .24, and .24. Thus, there were large total effects of Parental Education on mathematics, 
science, and reading achievement, and the total indirect effects also were substantial. The 
indirect effects were mediated via the Main Path and via Books. There also was a direct effect 
on Activity of Parental Education. In homes with a high level of activity, numeracy skills 
were assessed higher than literacy skills, which caused positive indirect effects of Activity on 
achievement in all three domains. 



GENDER The total effects of Gender were -.02, -.01, and .08 for mathematics, science, and 
reading, respectively, and the corresponding total indirect effects were .01, .01, and .02. The 
small indirect effect was partially mediated via Books and via Ability. For girls, there also was 
a stronger emphasis on literacy than on numeracy activity, which had a positive indirect effect 
on achievement via Ability, but also negative direct effects in all three domains. 
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Exhibit 4.41: The Russian Federation 
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PARENTAL EDUCATION The total effects of Parental Education were .27, .27, and .29 for 
mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were .1 1, .13, and .13. The indirect effects were primarily mediated via the Main Path and via 
Books. There also was an effect of Parental Education on Ability, which had a positive effect 
on achievement in all three domains. 



GENDER The total effects of Gender were .01, -.01, and .14 for mathematics, science, and | 

c 

reading, respectively, and the corresponding total indirect effects were .02, .02, and .03. Girls | 
thus outperformed boys in reading. A part of the indirect effects was mediated via Activity | 
and Ability. For girls, there also was more emphasis on literacy activities than on numeracy | 
activities, which had a positive impact on achievement via Ability. | 

o 
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Exhibit 4.42: Saudi Arabia 
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PARENTAL EDUCATION The total effects of Parental Education were .18, .25, and .24 for 
mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were .09, .10, and .09. The indirect effects were primarily mediated via the Main Path and 
via Books. There also was greater emphasis on literacy activities than numeracy activities in 
homes with higher Parental Education, which influenced achievement in all three domains 
via was Ability. 



GENDER The total effects of Gender were .06, .20, and .27 for mathematics, science, and reading, 
respectively, and the corresponding total indirect effects were .04, .04, and .04. Girls thus 
outperformed boys in all three domains, but particularly so in reading and science. For girls, 
there was a higher level of Activity and higher level of Ability, which was positively related to 
achievement in all three domains. 
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Exhibit 4.43 Singapore 


TIMSS &PIRLSI 
201ll 


PARENTAL EDUCATION The total effects of Parental Education were .39, .44, and .41 for 
mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were .15, .17, and .17. The indirect effects were mainly mediated via the Main Path and via 
Books. There also was an effect of Books on the balance of activities, such that in homes with 
many books there was greater emphasis on literacy activities than on numeracy activities. This 
had a weak effect on Ability, which influenced achievement in all three domains positively. 



GENDER The total effects of Gender were .02, -.03, and .10 for mathematics, science, and 
reading, respectively, and the corresponding total indirect effects were .03, .04, and .04. Girls 
thus outperformed boys in reading, while boys had a somewhat higher level of achievement 
in science. The indirect effects were partially mediated via the Main Path and via Ability. For 
girls, there also was more of an emphasis on literacy activities than numeracy activities, which 
had a positive effect on achievement via Ability, and a negative direct effect on mathematics 
achievement. Girls also had somewhat higher assessed literacy skills than numeracy skills, 
which had a direct effect on reading achievement. 
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Exhibit 4.44: The Slovak Republic 
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PARENTAL EDUCATION The total effects of Parental Education were .37, .38, and .38 for 
mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were .21, .21, and .21. The indirect effects were mediated via the Main Path and via Books. 
For homes with a high level of parental education, there was more emphasis on numeracy 
activities than on literacy activities, which had a weak negative effect on achievement via 
Ability. 



GENDER The total effects of Gender were -.05, -.05, and .08 for mathematics, science, and 
reading, respectively, and the corresponding total indirect effects were .00, .01, and .01. The 
weak indirect effects were mediated the Main Path and via Activity. There also was an indirect 
effect via Books. For girls, there also was more of an emphasis on literacy activities than 
numeracy activities. This had a positive effect on achievement, which was mediated via Ability. 
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Exhibit 4.45: Slovenia 
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PARENTAL EDUCATION The total effects of Parental Education were .38, .39, and .35 for 
mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were .15, .17, and .17. The indirect effects were mediated via the Main Path and via Books. 
In homes with more highly educated parents, there was more emphasis on literacy activities 
than on numeracy activities. This had a positive indirect effect on achievement, which was 
mediated via Ability. 



GENDER The total effects of Gender were -.06, -.02, and .12 for mathematics, science, and | 
reading, respectively, and the corresponding total indirect effects were .04, .05, and .06. | 
Girls thus outperformed boys in reading, while boys had a higher level of achievement in | 

O 

mathematics. The indirect effects were mediated via the Main Path and via Ability. There 1/1 
also was an indirect effect via Books. For girls, there was more emphasis on literacy activities 
than numeracy activities, which had a positive indirect effect on achievement in all domains 
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Exhibit 4.46: Spain 
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PARENTAL EDUCATION The total effects of Parental Education were .37, .33, and .31 for 
mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were .17, .19, and .16. The indirect effects were mediated via the Main Path and via Books. In 
homes with a larger number of books, there was greater emphasis on literacy activities than 
numeracy activities, which had a positive effect on achievement in all domains via Ability. 



GENDER The total effects of Gender were -.08, -.07, and .03 for mathematics, science, and 
reading, respectively, and the corresponding total indirect effects were .01, .02, and .03. Boys 
thus outperformed girls in mathematics and science, while girls had a somewhat higher level 
of achievement in reading. For girls, there was more emphasis on literacy than on numeracy 
activities, which had a positive indirect effect on achievement in all three domains via Ability, 
along with a positive direct effect on reading achievement. Girls also were assessed higher on 
literacy skills than on numeracy skills, which had a positive effect on reading achievement 
in particular. 
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Exhibit 4.47: Sweden 
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PARENTAL EDUCATION The total effects of Parental Education were .32, .34, and .34 for 
mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were .20, .25, and .22. The indirect effects were mediated via the Main Path and via Books. 
Both for homes with more books and more highly educated parents there was more literacy 
activity than numeracy activity. This had a positive indirect effect on achievement in all three 
domains, which was mediated via Ability, and there also was a positive direct effect on reading 
achievement. 



GENDER The total effects of Gender were -.05, -.03, and .11 for mathematics, science, and | 
reading, respectively, and the corresponding total indirect effects were .05, .08, and .10. | 
Girls thus outperformed boys in reading, while boys had a higher level of achievement in | 

O 

mathematics and science. Weak indirect effects of Gender were mediated via Books and 1/1 
via Ability. For girls, there also was more emphasis on literacy activities than on numeracy 
activities, which had a positive indirect effect on achievement via Ability, and there was also 
a positive direct effect on reading achievement. Girls also were more highly assessed on 
literacy skills than on numeracy skills, which was associated with a higher level of reading 
achievement. 
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Exhibit 4.48: United Arab Emirates 
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PARENTAL EDUCATION The total effects of Parental Education were .39, .40, and .42 for 
mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were .14, .13, and .15. The indirect effects were mediated via the Main Path and via Books. 
Homes with many books also tended to put more emphasis on literacy activities than 
numeracy activities, which indirectly had a positive effect on achievement via Ability. Parents 
with a higher level of education tended to assess literacy skills higher than numeracy skills, 
which also was positively related to achievement in all three domains. 



GENDER The total effects of Gender were .04, .09, and .14 for mathematics, science, and reading, 
respectively, and the corresponding total indirect effects were .01, .02, and .02. Girls thus 
outperformed boys in all three domains, and particularly so in reading and science. For 
girls, there was more emphasis on literacy activity than numeracy activity which indirectly 
influenced achievement positively via Ability. For students who had more of literacy than 
numeracy activities, literacy skills also were assessed higher than numeracy skills, which also 
had positive effects on achievement in all three domains. 
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Exhibit 4.49: Botswana sixth Grade country 
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PARENTAL EDUCATION The total effects of Parental Education were .41, .45, and .48 for 
mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were .09, .10, and .10. A part of the indirect effect was mediated via the Main Path, but there 
also was a relatively strong direct effect of Parental Education on Activity, and also a smaller 
direct effect of Parental Education on Ability. 



to 

c 

o 

(O 

c 

GENDER The total effects of Gender were .10, .06, and .15 for mathematics, science, and reading, | 
respectively, and the corresponding total indirect effects were .02, .02, and .01. Thus, girls | 
outperformed boys in all three domains. Most of the effect of Gender was direct. However, | 
for girls there was more emphasis on literacy activity than on numeracy activity, which had | 

O 

a positive effect on achievement in all three domains via Ability. 
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Exhibit 4.50: Honduras Sixth Grade country 
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PARENTAL EDUCATION The total effects of Parental Education were .34, .36, and .34 for 
mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were .03, .05, and .05. The total indirect effect thus accounted for only a small part of the total 
effect. Small indirect effects went via the Main Path and via Books. To a small extent, the effect 
of Parental Education was mediated via the balance of the activities and assessment of literacy 
and numeracy skills, as well as more highly educated parents tending to emphasize numeracy 
activities more and to assess numeracy skills higher. 



GENDER The total effects of Gender were -.08, -.06, and .07 for mathematics, science, and 
reading, respectively, and the total indirect effects were all close to 0. Girls thus outperformed 
boys in reading, while boys had higher achievement in mathematics and science. However, 
no significant indirect effects were identified. There was, however, a tendency for girls to 
have more emphasis on literacy activities than on numeracy activities, which influenced 
achievement in all three domains positively via Ability. 
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SOURCE: lEA's Trends in International Mathematics and Science Study and Progress in International Reading Literacy Study - TIMSS and PIRLS 201 1 




Exhibit 4.51 : Quebec, Canada Benchmarking Participant 
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PARENTAL EDUCATION The total effects of Parental Education were .40, .39, and .40 for 
mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were . 13, . 12, and .15. The indirect effects were mediated via the Main Path and via Books. In 
families with a larger number of books, there was greater emphasis on literacy than numeracy 
activities, which had positive indirect effects on achievement via Ability, and there also was a 
positive direct effect on reading achievement. 



GENDER The total effects of Gender were .08, .14, and .19 for mathematics, science, and reading, 
respectively, and the corresponding total indirect effects were .03, .03, and .03. Girls thus 
outperformed boys in all three domains, and particularly so in reading and science. For girls, 
there was more emphasis on literacy activities than on numeracy activities, which had positive 
indirect effects on all domains of achievement via Ability, and also a positive direct effect on 
reading achievement. 
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Exhibit 4.52: Abu Dhabi, UAE Benchmarking Participant 
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PARENTAL EDUCATION The total effects of Parental Education were .40, .39, and .40 for 
mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were .13, .12, and .15. The indirect effects were mediated via the Main Path and via Books. 
In families with a larger number of books, more emphasis was placed on literacy activities 
than on numeracy activities, which had a positive indirect effect on achievement via Ability. 
Parents with a higher level of education also assessed literacy skills higher than numeracy 
skills, which influenced achievement in all three domains. 



GENDER The total effects of Gender were .08, .14, and .19 for mathematics, science, and reading, 
respectively, and the corresponding total indirect effects were .03, .03, and .03. Girls thus 
outperformed boys in all three domains, and particularly so in reading. The indirect effects 
of Gender were mediated via Activity and via Ability. For girls, there was a stronger emphasis 
on literacy activities than on numeracy activities, which had a positive indirect effect on 
achievement via Ability. 
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SOURCE: lEA's Trends in International Mathematics and Science Study and Progress in International Reading Literacy Study - TIMSS and PIRLS 201 1 




Exhibit 4.53: Dubai, UAE Benchmarking Participant 


TIMSS fr PIRLS 
2011 


^th 

Grade 


PARENTAL EDUCATION The total effects of Parental Education were .41, .42, and .42 for 
mathematics, science, and reading, respectively, and the corresponding total indirect effects 
were .16, .16, and .17. Indirect effects were mediated via the Main Path and via Books. In 
homes with many books, more emphasis was placed on literacy activities than numeracy 
activities, which had an indirect effect on achievement in all domains via Ability. Parents with 
a higher level of education also assessed literacy skills higher than numeracy skills, which was 
associated with a higher level of achievement in all three domains. 



T3 

c 
ai 
I — 

GENDER The total effects of Gender were -.01, .02, and .07 for mathematics, science, and | 
reading, respectively, and the corresponding total indirect effects were .03, .04, and .04. Girls | 

O 

thus outperformed boys in reading. Indirect effects were mediated via the Main Path and via 
Books. For girls, more emphasis was placed on literacy activities than on numeracy activities; 
this had an indirect effect on achievement via Ability, and via a higher assessment of literacy 
skills than numeracy skills. 



TIMSS AND PIRLS 201 1 RELATIONSHIPS REPORT 

284 CHAPTER 4 



TIMSS & PIRLS 

International Study Center 

Lynch School of Education, Boston College 




References 

Aiken, L.R. (1972). Language factors in 
learning mathematics. Review of 
Educational Research, 42, 359-385. 

Anders, Y., Rossbach, H.-G., Weinert, S., 
Ebert, S., Kuger, S., Lehrl, S., & von 
Maurice, J. (2012). Home and preschool 
learning environments and their 
relations to the development of early nu- 
meracy skills. Early Childhood Research 
Quarterly, 27, 231-244. 

Baker, D.R & Jones, D.P. (1993). Creating 
gender equality: Cross-national gender 
stratification and mathematical 
performance. Sociology of Education, 66, 
91-103. 

Bandura, A. (1986). Social foundations of 
thought and action: A social cognitive 
theory. Englewood Cliffs, NJ: Prentice- 
Hall. 

Barone, C. (2006). Cultural capital, ambition 
and the explanation of inequalities in 
learning outcomes: a comparative 
analysis. Sociology, 40(6), 1039-1058. 

Bernstein, B. (1971). Social class, language 
and socialization. In B. Bernstein (Ed.), 
Class, codes and control. London: 
Routledge & Kegan Paul. 

Brooks-Gunn, J., Klebanov, P., & Duncan, D. 
(1996). Ethnic differences in children’s 
intelligence test scores: Role of economic 
deprivation, home environment, and 
maternal characteristics. Child 
Development, 67, 396-408. 

Brown, T.A. (2006) Confirmatory factor 
analysis for applied research. New York, 
NY: Guilford Press. 

Bourdieu, P. & Passeron, J.-C. (1977). 
Reproduction in education, society and 
culture. Beverly Hills: Sage. 

Davis-Kean, PE. (2005). The influence of 
parent education and family income on 
child achievement: the indirect role of 
parental expectations and the home 
environment. Journal of Family 
Psychology, 19(2), 294-304. 


De Graaf, N.D., De Graaf, P.M., & 

Kraaykamp, G. (2000). Parental cultural 
capital and educational attainment in 
the Netherlands: A refinement of the 
cultural capital perspective. Sociology of 
Education, 73, 92-111. 

Diezman, C. & Yelland, N. (2000). 

Developing mathematical literacy in the 
early childhood years. In N.J. Yelland 
(Ed.), Promoting meaningful learning. 
Washington, DC: National Association 
for the Education of Young Children. 

Doig, B., McCrae, B., & Rowe, K. (2003). A 
good start to numeracy. Effective 
numeracy strategies from research and 
practice in early childhood. Camberwell, 
Victoria: Australian Council for 
Educational Research. 

Duncan, G., Brooks-Gunn, J., & Klebanov, P. 
(1994). Economic deprivation and early 
child development. Child Development, 
65, 296-318. 

Eccles, J.S. (1994). Understanding women’s 
educational and occupational choices: 
Applying the Eccles et al. model of 
achievement-related choices. Psychology 
of Women Quarterly, 18, 585-610. 

Else-Quest, N.M., Hyde, J.S., & Linn, M.C. 
(2010). Cross-national patterns of gender 
differences in mathematics achievement, 
attitudes, & affect: A meta-analysis. 
Psychological Bulletin, 136, 103-127. 

Ewers-Rogers, J. & Cowan, R. (1996). 

Children as apprentices to number. Early 
Childhood Development and Care, 125, 
15-25. 

Farkas, G. & Beron, K. (2004). The detailed 
age trajectory of oral vocabulary 
knowledge: Differences by class and race. 
Social Science Research, 33, 464-497. 

Gustafsson, J.-E. (2002). Measurement from 
a hierarchical point of view. In H. I. 
Braun, D.N. Jackson & D.E. Wiley (Eds.), 
The role of constructs in psychological and 
educational measurement (pp. 73-95). 
London: Lawrence Erlbaum Associates, 
Publishers. 



TIMSS & PIRLS 

International Study Center 

Lynch School of Education, Boston College 


EFFECTS OF HOME BACKGROUND ON STUDENT ACHIEVEMENT IN 
READING, MATHEMATICS, AND SCIENCE AT THE FOURTH GRADE 

CHAPTER 4 


285 



Gustafsson, J.-E. & Balke, G. (1993). General 
and specific abilities as predictors of 
school achievement. Multivariate 
Behavioral Research, 28(4), 407-434. 

Gustafsson, J.-E. & Aberg-Bengtsson, L. 
(2010). Unidimensionality and 
interpretability of psychological 
instruments. In S.E. Embretson (Ed.), 
Measuring psychological constructs: 
Advances in model-based approaches. 
Washington: American Psychological 
Association. 

Hart, B. & Risley, R.T. (1995). Meaningful 
differences in the everyday experience 
of young American children. Baltimore, 
MD: Brookes. 

Hecht, S.A., Burgess, S.R., Torgesen, J.K., 
Wagner, R.K., & Rashotte, C.A. (2000). 
Explaining social class differences in 
growth of reading skills from beginning 
kindergarten through fourth-grade: The 
role of phonological awareness, rate of 
access, and print knowledge. Reading 
and Writing: An Interdisciplinary Journal, 
12, 99-127. 

Hyde, J.S. (2005). The gender similarities 
hypothesis. American Psychologist, 60, 
581-592. 

Jordan, G.E., Snow, C.E., & Porsche, M.V. 
(2000). Project EASE: The effect of a 
family literacy project on kindergarten 
students’ early literacy skills. Reading 
Research Quarterly, 35, 524-546. 

Lyster, S.-A.H. (2002). The effects of 
morphological versus phonological 
awareness training in kindergarten 
on reading development. Reading and 
Writing: An Interdisciplinary Journal, 15, 
261-294. 

Martin, M. O., Mullis, I. V. S., Foy, P, & 
Arora, A. (2012). Creating and 
interpreting the TIMSS and PIRLS 2011 
context questionnaire scales. In M.O. 
Martin & I.V.S. Mullis (Eds.), Methods 
and Procedures in TIMSS and PIRLS 
2011. Chestnut Hill, MA: TIMSS & 
PIRLS International Study Center, Lynch 


School of Education, Boston College. 
Retrieved from http://timssandpirls. 
bc.edu/methods/pdf/TPll_Context_Q_ 
Scales.pdf 

Ming Chui, M. & McBride-Chang, C. 

(2006). Gender, context, and reading: 

A comparison of students in 43 
countries. Scientific Studies of Reading, 
10(4), 331-62. 

Mullis, I.V.S., Martin, M.O., Fierros, E.G., 
Goldberg, A.L., & Stemler, S.E. (2000). 
Gender differences in achievement: IEA’s 
third international mathematics and sci- 
ence study. Chestnut Hill, MA: TIMSS & 
PIRLS International Study Center, Lynch 
School of Education, Boston College. 

Mullis, I.V.S., Martin M.O., Gonzalez E.J., & 
Kennedy, A.M. (2003). PIRLS 2001 
international report: IEA’s study of 
reading literacy achievement in primary 
schools in 35 countries. Chestnut Hill, 
MA: TIMSS & PIRLS International 
Study Center, Lynch School of Educa- 
tion, Boston College. 

Mullis, I.V.S., Martin M.O., Kennedy A.M., 
& Foy, P. (2007). PIRLS 2006 
international report: IEA’s progress in 
international reading literacy study in 
primary schools in 40 countries. Chestnut 
Hill, MA: TIMSS & PIRLS International 
Study Center, Lynch School of Educa- 
tion, Boston College. 

Mullis, I.V.S., Martin M.O., Foy, P, & 
Drucker, K.T. (2012). PIRLS 2011 
international results in reading. Chestnut 
Hill, MA: TIMSS & PIRLS International 
Study Center, Lynch School of Educa- 
tion, Boston College. 

Muthen, L.K. & Muthen, B.O. (1998-2012). 
Mplus user’s guide. Seventh Edition. Los 
Angeles, CA: Muthen & Muthen. 

Myrberg, E. & Rosen, M. (2009). Direct and 
indirect effects of parents’ education on 
reading achievement among third 
graders in Sweden. British Journal of 
Educational Psychology, 79, 695-711. 


TIMSS AND PIRLS 201 1 RELATIONSHIPS REPORT 

286 CHAPTER 4 


TIMSS & PIRLS 

International Study Center 

Lynch School of Education, Boston College 


National Early Literacy Panel. (2008). 
Developing early literacy: Report of the 
national early literacy panel. Washing- 
ton, DC: National Institute for Literacy. 
Retrieved from http://lincs.ed.gov/publi- 
cations/pdf/NELP Report09.pdf 

Noble, K.G., McCandless, B.D., & Parah, 

M.J. (2007). Socioeconomic gradients 
predict individual differences in neuro- 
cognitive development. Developmental 
Science, 10, 464-480. 

Park, H. (2008) Home literacy environments 
and children’s reading performance: 
a comparative study of 25 countries. 
Educational Research and Evaluation; 

An international Journal of Theory and 
Practice, 14(6), 489-505. 

Raz, I.S. & Bryant, P. (1990). Social 
background, phonological awareness 
and children’s reading. British Journal of 
Developmental Psychology, 8, 209-225. 

Reise, S.P. (2012). The rediscovery of 
bifactor measurement models. 
Multivariate Behavioral Research, 47(5), 
667-696. 

Senechal, M. & LePevre, J.A. (2002). 

Parental involvement in the development 
of children’s reading skill: A five-year 
longitudinal study. Child Development, 
73, 445-460. 


Sirin, S.R. (2005). Socioeconomic status and 
academic achievement: A Meta-analytic 
review of research 1990-2000. Review of 
Educational Research, 75(3), 417-45. 

Snow, C.E., Burns, M.S., & Griffin, P. (Eds.). 
(1998). Preventing reading difficulties in 
young children. Washington, DC: 
National Academy Press. 

Spelke, E.S. (2005). Sex differences in 
intrinsic aptitude for mathematics and 
science? American Psychologist, 60, 
950-958. 

Tabors, P.O., Snow, C.E., & Dickinson, D.K. 
(2001). Homes and schools 
together: Supporting language and 
literacy development. In D.K. Dickinson 
& P.O. Tabors (Eds.), Beginning literacy 
with language: Young children learning 
at home and in school (pp. 313-334). 
Baltimore: Brookes. 

White, K.R. (1982). The relation between 
socio-economic status and academic 
achievement. Psychological Bulletin, 
91(3), 461-481. 

Yang, Y. (2003). Measuring socio-economic 
status and its effects on individual and 
collective levels: A cross-country 
comparison. Gothenburg: Gothenburg 
Studies in Educational Sciences, Acta 
Universitatis Gothoburgensis. 



TIMSS & PIRLS 

International Study Center 

Lynch School of Education, Boston College 


EFFECTS OF HOME BACKGROUND ON STUDENT ACHIEVEMENT IN 
READING, MATHEMATICS, AND SCIENCE AT THE FOURTH GRADE 

CHAPTER 4 


287 


TIMSS & PIRLS 

International Study Center 

Lynch School of Education, Boston College 

typography: Set in Avant Garde Gothic, Meridien, Minion, and Myriad. 
cover design: Ruthanne Ryan 
cover illustration: Steven A. Simpson 
book design: Mario A. Pita and Ruthanne Ryan 

layout & production: Susan Farrell, Jennifer Moher Sepulveda, Mario A. Pita, and Steven A. Simpson 
director, graphic design & publications: Paul Connolly 







TIMSS & PIRLS 

International Study Center 

Lynch School of Education, Boston College 



BOSTON 

COLLEGE 




timssandpirls.bc.edu 

Copyright © 201 3 International Association for the 
Evaluation of Educational Achievement (IEA) 


ISBN: 978-1-889938-18-9 




